Memory System Topologies Including A Memory Die Stack

ABSTRACT

Systems, among other embodiments, include topologies (data and/or control/address information) between an integrated circuit buffer device (that may be coupled to a master, such as a memory controller) and a plurality of integrated circuit memory devices. For example, data may be provided between the plurality of integrated circuit memory devices and the integrated circuit buffer device using separate segmented (or point-to-point link) signal paths in response to control/address information provided from the integrated circuit buffer device to the plurality of integrated circuit buffer devices using a single fly-by (or bus) signal path. An integrated circuit buffer device enables configurable effective memory organization of the plurality of integrated circuit memory devices. The memory organization represented by the integrated circuit buffer device to a memory controller may be different than the actual memory organization behind or coupled to the integrated circuit buffer device. The buffer device segments and merges the data transferred between the memory controller that expects a particular memory organization and actual memory organization.

RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.16/842,368, filed on Apr. 7, 2020, which is a continuation of U.S.patent application Ser. No. 16/692,043, filed on Nov. 22, 2019 (now U.S.patent Ser. No. 10/672,458), which is a continuation of U.S. patentapplication Ser. No. 16/214,986, filed on Dec. 10, 2018 (now U.S. patentSer. No. 10/535,398), which is a continuation of U.S. patent applicationSer. No. 15/832,468 filed on Dec. 5, 2017 (now U.S. patent Ser. No.10/381,067), which is a continuation of U.S. patent application Ser. No.15/389,409 filed on Dec. 22, 2016 (now U.S. Pat. No. 9,865,329), whichis a continuation of U.S. patent application Ser. No. 14/801,723 filedon Jul. 16, 2015 (now U.S. Pat. No. 9,563,583), which is a continuationof U.S. patent application Ser. No. 14/015,648 filed on Aug. 30, 2013(now U.S. Pat. No. 9,117,035), which is a continuation of U.S. patentapplication Ser. No. 13/149,682 filed on May 31, 2011 (now U.S. Pat. No.8,539,152), which is a continuation of U.S. patent application Ser. No.12/703,521 filed on Feb. 10, 2010 (now U.S. Pat. No. 8,108,607) which isa continuation of U.S. patent application Ser. No. 12/424,442 filed onApr. 15, 2009 (now U.S. Pat. No. 7,685,364) which is a divisional ofU.S. patent application Ser. No. 11/697,572 filed on Apr. 6, 2007 (nowU.S. Pat. No. 7,562,271) which is a continuation-in-part of U.S. patentapplication Ser. No. 11/460,899 filed on Jul. 28, 2006 (now U.S. Pat.No. 7,729,151) which is a continuation-in-part of U.S. patentapplication Ser. No. 11/236,401 filed on Sep. 26, 2005 (now U.S. Pat.No. 7,464,225).

FIELD OF THE INVENTION

The present invention generally relates to integrated circuit devices,high speed signaling of such devices, memory devices, and memorysystems.

BACKGROUND

Some contemporary trends predict that processors, such as generalpurpose microprocessors and graphics processors, will continue toincrease system memory and data bandwidth requirements. Usingparallelism in applications such as multi-core processor architecturesand multiple graphics pipelines, processors should be able to driveincreases in system bandwidths at rates some predict will be doubledevery three years for the next ten years. There are several major trendsin dynamic random access memory (“DRAM”) that may make it costly andchallenging to keep up with increasing data bandwidth and system memoryrequirements. For example, transistor speed relative to feature sizeimprovements in a given DRAM technology node, and the rising costs ofcapital investment required to move DRAM technology to greater memorydensities for a given DRAM die adversely affect the rate at which DRAMtechnology can keep pace with the increasing data bandwidth and systemcapacity requirements.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments are illustrated by way of example, and not by way oflimitation, in the figures of the accompanying drawings and in whichlike reference numerals refer to similar elements and in which:

FIG. 1 illustrates a memory module topology including a plurality ofintegrated circuit memory devices and a plurality of integrated circuitbuffer devices;

FIG. 2 illustrates a memory module topology having a split multi-dropcontrol/address bus;

FIG. 3 illustrates a memory module topology having a single multi-dropcontrol/address bus;

FIG. 4 illustrates a memory module topology that provides data betweeneach integrated circuit buffer device and a memory module connectorinterface;

FIG. 5 illustrates a memory module topology including a plurality ofintegrated circuit memory devices and a plurality of integrated circuitbuffer devices with an integrated circuit buffer device for control andaddress information;

FIG. 6 illustrates termination of a control/address signal path in amemory module topology of FIG. 5;

FIG. 7 illustrates termination of data signal paths in a memory moduletopology of FIG. 5;

FIG. 8 illustrates termination of a split control/address signal path ina memory module topology of FIG. 5;

FIG. 9A illustrates a top view of a memory module topology including aplurality of integrated circuit memory devices and a plurality ofintegrated circuit buffer devices;

FIG. 9B illustrates a side view of a memory module topology including aplurality of integrated circuit memory devices and a plurality ofintegrated circuit buffer devices;

FIG. 9C illustrates a bottom view of a memory module topology includinga plurality of integrated circuit memory devices and a plurality ofintegrated circuit buffer devices;

FIG. 10 is a block diagram illustrating a topology of a device having aplurality of integrated circuit memory dies and an integrated circuitbuffer die;

FIG. 11 illustrates a multi-chip package (“MCP”) device having aplurality of integrated circuit memory dies and an integrated circuitbuffer die;

FIG. 12 illustrates a device having a plurality of integrated circuitmemory dies and a buffer die;

FIG. 13 illustrates a device having a plurality of integrated circuitmemory devices and a buffer device that are disposed on a flexible tape;

FIG. 14 illustrates a device having a plurality of integrated circuitmemory dies and a buffer die that are disposed side-by-side and housedin a package;

FIG. 15 illustrates a device having a plurality of integrated circuitmemory dies and a buffer die that are housed in separate packages andintegrated together into a larger package-on-a-package (“POP”) device;

FIG. 16 illustrates a memory module topology including a serial presencedetect device (“SPD”);

FIG. 17 illustrates a memory module topology with each data slice havingan SPD;

FIG. 18 is a block diagram of an integrated circuit buffer die;

FIG. 19 is a block diagram of a memory device;

FIGS. 20A-B illustrate signal paths between memory module interfaceportions and a plurality of integrated circuit buffer devices;

FIGS. 21A-D illustrate memory system point-to-point topologies includinga master and at least one memory module (shown as buffer 101 a) having aplurality of integrated circuit memory devices;

FIGS. 22A-C illustrate memory system daisy chain topologies including amaster and at least one memory module having a plurality of integratedcircuit memory devices;

FIGS. 23A-C and 24A-B illustrate memory system topologies including amaster to provide control/address information to a plurality ofintegrated circuit buffer devices;

FIGS. 25A-B illustrate memory modules having different sized addressspaces, or memory capacity;

FIGS. 26A-B illustrate a memory system including a master and two memorymodules operating during a first and second mode of operation (bypassmode);

FIG. 27 illustrates a memory system including a master and at least fourmemory modules;

FIGS. 28A-B illustrate memory systems including a master and four memorymodules operating during a first mode and second mode of operation(bypass mode);

FIG. 29 illustrates a bypass circuit;

FIGS. 30A-B illustrate timing diagrams for an integrated circuit bufferdevice;

FIG. 31 illustrates a method to levelize memory modules according to anembodiment;

FIGS. 32A-E illustrate tree topologies (data and/or control/addressinformation) between an integrated circuit buffer device and a pluralityof integrated circuit memory devices;

FIGS. 33A-B illustrate fly-by topologies (data and/or control/addressinformation) between an integrated circuit buffer device and a pluralityof integrated circuit memory devices;

FIG. 34 illustrates point-to-point (also known as segmented) topology(data and/or control/address information) between an integrated circuitbuffer device and a plurality of integrated circuit memory devices;

FIG. 35 illustrates an MCP (or system-in-a-package (“SIP”) topology(data and/or control/address information) between an integrated circuitbuffer die and a plurality of integrated circuit memory dies;

FIG. 36 is a block diagram of an integrated circuit buffer device;

FIGS. 37A-B illustrate timing diagrams of an integrated circuit bufferdevice;

FIG. 38 illustrates a buffer device and a plurality of integratedcircuit memory devices in different ranks;

FIG. 39 illustrates a system for accessing individual memory devicesthat function as respective memory ranks;

FIG. 40 illustrates a method of operation in an integrated circuitbuffer device.

DETAILED DESCRIPTION

Systems, among other embodiments, include topologies for transferringdata and/or control/address information between an integrated circuitbuffer device (that may be coupled to a master, such as a memorycontroller) and a plurality of integrated circuit memory devices. Forexample, data may be provided between the plurality of integratedcircuit memory devices and the integrated circuit buffer device usingseparate segmented (or point-to-point link) signal paths in response tocontrol/address information provided from the integrated circuit bufferdevice to the plurality of integrated circuit buffer devices using asingle fly-by (or bus) signal path. Other topology types may includeforked, star, fly-by, segmented and topologies used in SIP or MCPembodiments.

An integrated circuit buffer device enables configurable effectivememory organization of a plurality of integrated circuit memory devices.The memory organization represented by the integrated circuit bufferdevice to a memory controller may be different than the actual memoryorganization behind or coupled to the integrated circuit buffer device.For example, control/address information may be provided to the bufferdevice from a memory controller that expects a memory organizationhaving a predetermined number of memory devices and memory banks as wellas page size and peak bandwidth, but the actual memory organizationcoupled to the buffer device is different. The buffer device segmentsand/or merges the data transferred between the memory controller thatexpects a particular memory organization and the actual memoryorganization. The integrated circuit buffer device may merge read datafrom separate memory devices into a stream of read data. Likewise, theintegrated circuit memory device may segment a write data into writedata portions that are stored on a plurality of memory devices.

An integrated circuit buffer device may include data path, addresstranslation, data path router, command decode and control (or registerset) circuits. The buffer device also includes an interface that may beconfigured into at least three different segmentation modes: 1) Four4-bit interfaces (4×4), 2) Two 4-bit interfaces (2×4) or 3) Two 8-bitinterfaces (2×8). The different configurations allow flexibility inmemory module or memory stack configurations. The buffer device may alsoinclude a pattern generator and internal memory array circuit to emulatestoring and retrieving data from the plurality of integrated circuitmemory devices.

The buffer device may increase memory system performance by; forexample, eliminating a “time bubble” or idle time for a signal path(bus) turnaround between memory transactions to different ranks ofintegrated circuit memory devices coupled to segmented data signalpaths. A memory rank may also include a single integrated circuit memorydevice. Eliminating the need for the memory controller to track memoryrank access and inserting time bubbles may reduce memory controllercomplexity. Memory modules or memory rank capacity may be expanded usingsegmented data signal paths without decreasing bandwidth caused bybubble time insertion. Memory modules may include more memory devices ordies while still emulating a single rank memory module.

According to embodiments, a system includes a master device and a firstmemory module having a plurality of integrated circuit memory devicesand a plurality of integrated circuit buffer devices that operate infirst and second modes of operation (bypass mode). In a first mode ofoperation, a first memory module provides read data from the pluralityof integrated circuit memory devices (via an integrated circuit bufferdevice) on a first signal path to the master and a second memory modulesimultaneously provides read data from its plurality of integratedcircuit memory devices (via another integrated circuit buffer device onthe second module) on a third signal path coupled to the master device.In a second mode of operation, the first memory module provides firstread data from its plurality of integrated circuit memory devices (viathe integrated circuit buffer device) on the first signal path andsecond read data from its plurality of integrated circuit memory devices(via the integrated circuit buffer device) on a second signal path thatis coupled to a second memory module. An integrated circuit bufferdevice in the second memory module then bypasses the second read datafrom the second signal path and provides the second read data on a thirdsignal path coupled to the master device. The first memory module mayhave a larger address space or capacity, such as twice as large, ascompared to the second memory module.

Similarly, write data may be provided from the master device to thefirst and second memory modules during the first and second modes ofoperation.

According to embodiments, the second memory module includes a bypasscircuit (such as in the integrated circuit buffer device, interface orin continuity memory module) to transfer the second read data from thesecond signal path to the third signal path. The bypass circuit mayinclude a jumper, signal trace and/or semiconductor device. The bypasscircuit may also include delay circuits for adding delay in outputtingthe read data (or levelizing) from a memory module.

According to embodiments, a system includes a master device and at leastfour memory modules wherein at least two memory modules have differentcapacities than the other two memory modules. The four memory modulesare coupled to a plurality of signal paths. The system may operate in abypass mode in which one or more memory modules use a bypass circuit toprovide read data from at least one larger capacity memory module to amaster device.

According to embodiments, a system includes a master and a plurality ofmemory modules that may be disposed in a variety of topologies, such aspoint-to-point or daisy chain topologies. Memory modules may include aplurality of integrated circuit buffer devices that are coupled using avariety of topologies to receive control information, such as dedicated,fly-by, stub, serpentine or tree topologies, singly or in combination.

According to embodiments, a method determines a mode of operation of asystem including a master and a plurality of memory modules. In a bypassmode of operation, delays are provided to read data from at least onememory module to levelize or ensure that read data from differentcapacity memory modules using different signal paths arrive at themaster at approximately the same time.

According to embodiments, a memory module includes a plurality of signalpaths that provide data to a memory module connector from a plurality ofrespective integrated circuit buffer devices (or dies) that access thedata from an associated plurality of integrated circuit memory devices(or dies). In a specific embodiment, each integrated circuit bufferdevice is also coupled to a bussed signal path that provides controland/or address information that specifies an access to at least oneintegrated circuit memory device associated with the respectiveintegrated circuit buffer device.

According to embodiments, a memory module connector includes acontrol/address interface portion and a data interface portion. Acontrol/address bus couples a plurality of integrated circuit bufferdevices to the control/address interface portion. A plurality of datasignal paths couple the plurality of respective integrated circuitbuffer devices to the data interface portion. Each integrated circuitbuffer device includes 1) an interface to couple to at least oneintegrated circuit memory device, 2) an interface to couple to thecontrol/address bus and 3) an interface to couple to a data signal pathin the plurality of data signal paths.

According to embodiments, a memory module may include a non-volatilememory location, for example using an electrically erasable programmableread only memory (“EEPROM”) (also known as a Serial Presence Detect(“SPD”) device), to store information regarding parameters andconfiguration of the memory module. In embodiments, at least oneintegrated circuit buffer device accesses information stored in the SPDdevice.

In a package embodiment, a package houses an integrated circuit bufferdie and the plurality of integrated circuit memory dies. In the package,a plurality of signal paths transfer data (read and/or write data)between the integrated circuit buffer die and the plurality ofintegrated circuit memory dies. The integrated circuit buffer dieprovides control signals from an interface of the package to theplurality of integrated circuit memory dies. Data stored in memoryarrays of the plurality of integrated circuit memory dies is provided toa signal path disposed on the memory module via the integrated circuitbuffer die in response to the control signals. In an embodiment, thepackage may be a multichip package (“MCP”). In an embodiment, theplurality of integrated circuit memory dies may be housed in common orseparate packages. In an embodiment described below, the memory modulemay include a series of integrated circuit dies (i.e., memory die andbuffer die) stacked on top of one another and coupled via a signal path.

As described herein, an integrated circuit buffer device is alsoreferred to as a buffer or buffer device. Likewise, an integratedcircuit memory device is also referred to as a memory device. A masterdevice is also referred to as a master.

In an embodiment, an integrated circuit memory device is distinguishedfrom a memory die in that a memory die is a monolithic integratedcircuit formed from semiconductor materials for storing and/orretrieving data or other memory functions, whereas an integrated circuitmemory device is a memory die having at least some form of packaging orinterface that allows the memory die to be accessed.

Likewise in an embodiment, an integrated circuit buffer device isdistinguished from a buffer die in that a buffer die is a monolithicintegrated circuit formed from semiconductor materials and performs atleast one or more buffer functions described herein, whereas anintegrated circuit buffer device is a buffer die having at least someform of packaging or interface that allows communication with the bufferdie.

In the embodiments described in more detail below, FIGS. 1-8 illustratecontrol/address and data signal path topologies including a plurality ofintegrated circuit memory devices (or dies) and a plurality ofintegrated circuit buffer devices (or dies) situated on a memory module.FIGS. 10, 18, and 19 also illustrate signal path topologies includingintegrated circuit memory devices (or dies) and integrated circuitbuffer devices (or dies) situated on a memory module as well as theoperation of an integrated circuit buffer device (or die) and memorydevice (or die) in embodiments among other things. FIGS. 21A-D, 22A-C,23A-C and 24A-B illustrate system topologies. FIGS. 26A-B, 28A-B and 31illustrate operating a memory system in a first and second mode ofoperation (bypass mode). FIGS. 32A-E, 33A-B, 34 and 35 illustratetopologies between an integrated circuit buffer device and a pluralityof integrated circuit memory devices. FIG. 36 is a block diagram of anintegrated circuit buffer device and FIGS. 37A-B illustrates timingdiagrams of an integrated circuit buffer device. FIGS. 38 and 39illustrate a buffer device and a plurality of integrated circuit memorydevices in different memory ranks. FIG. 40 illustrates a method ofoperation in an integrated circuit buffer device.

FIG. 1 illustrates a memory module topology including a plurality ofintegrated circuit memory devices and a plurality of associatedintegrated circuit buffer devices. In an embodiment, a memory module 100includes a plurality of buffer devices 100 a-d coupled to a commonaddress/control signal path 121. Each buffer device of the plurality ofbuffer devices 100 a-d provides access to a plurality of respectiveintegrated circuit memory devices 101 a-d via signal paths 102 a-d and103. In an embodiment, respective data slices a-d are formed by one ofbuffers 100 a-d and sets of memory devices 101 a-d. Each of bufferdevices 100 a-d is coupled to a respective set of signal paths 120 a-d,that transfer data (read and write data) between the buffer devices 100a-d and a memory module connector interface. In an embodiment, maskinformation is transferred to buffer devices 100 a-d from a memorymodule connector interface using signal paths 120 a-d, respectively.

In an embodiment, a data slice is a portion of the memory module datasignal path (or bus) that is coupled to the respective integratedcircuit buffer device. The data slice may include the full data path orportions of data paths to and from a single memory device disposed onthe memory module.

Integrated circuit memory devices may be considered as a common class ofintegrated circuit devices that have a plurality of storage cells,collectively referred to as a memory array. A memory device stores data(which may be retrieved) associated with a particular address provided,for example, as part of a write or read command. Examples of types ofmemory devices include dynamic random access memory (“DRAM”), includingsingle and double data rate synchronous DRAM, static random accessmemory (“SRAM”), and flash memory. A memory device typically includesrequest or command decode and array access logic that, among otherfunctions, decodes request and address information, and controls memorytransfers between a memory array and signal path. A memory device mayinclude a transmitter circuit to output data, for example, synchronouslywith respect to rising and falling edges of a clock signal, (e.g., in adouble data rate type of memory device). Similarly, the memory devicemay include a receiver circuit to receive data, for example,synchronously with respect to rising and falling edges of a clock signalor outputs data with a temporal relationship to a clock signal in anembodiment. A receiver circuit also may be included to receive controlinformation synchronously with respect to rising and falling edges of aclock signal. In an embodiment, strobe signals may accompany the datapropagating to or from a memory device and that data may be captured bya device (e.g., memory device or buffer, or controller) using the strobesignal.

In an embodiment, an integrated circuit buffer device is an integratedcircuit that acts as an interface between a memory module connectorinterface and at least one integrated circuit memory device. Inembodiments, the buffer device may store and/or route data, controlinformation, address information and/or a clock signal to at least oneintegrated circuit memory device that may be housed in a common orseparate package. In an embodiment, the buffer isolates, routes and/ortranslates data, control information and a clock signal, singly or incombination, between a plurality of memory devices and a memory moduleconnector interface. An embodiment of a memory module connectorinterface is described below and shown in FIGS. 9A-C.

At least one signal path 121, as shown in FIG. 1, disposed on memorymodule 100, transfers control and/or address (control/address)information between at least one of the buffer devices 100 a-d and amemory module connector interface in various embodiments. In anembodiment, signal path 121 is a multi-drop bus. As illustrated in FIGS.2-8 and described below, alternate topologies for transferringcontrol/address information, data and clock signals between one or morebuffer devices 100 a-d and a memory module connector interface may beused in alternate embodiments. For example, a split multi-dropcontrol/address bus, segmented multi-drop control/address bus, andpoint-to-point and/or daisy chain topologies for a data bus may beemployed.

In an embodiment, clock signals and/or clock information may betransferred on at least one signal line in signal path 121. These clocksignal(s) provide one or more clock signals having a known frequencyand/or phase. In an embodiment, a clock signal is synchronized with ortravels along side the control/address information. In an embodiment, anedge of the clock signal has a temporal relationship with an edge of acontrol/address signal representing the control/address information. Inan embodiment, a clock signal is generated by a clock source, masterdevice (e.g., controller device) and/or buffer device.

In an embodiment, a clock signal and/or clock information may betransferred on at least one signal line in respective signal paths 120a-d. Buffer devices 100 a-d may receive and/or transmit a clock signalwith data on signal paths 120 a-b. In an embodiment, write data isprovided to buffer devices 100 a-d on signal paths 120 a-d and a clocksignal is provided on signal paths 120 a-d along side write data. In anembodiment, a clock signal (such as a clock-to-master (“CTM”)) isprovided from buffer devices 100 a-d on signal paths 120 a-d along sideread data on signal paths 120 a-d. In an embodiment, a clock signal issynchronized with or travels along side the write and/or read data. Anedge of the clock signal has a temporal relationship or is aligned withan edge of a data signal representing write and/or read data. Clockinformation can be embedded in data, eliminating the use of separateclock signals along with the data signals.

In an embodiment, a read, write and/or bidirectional strobe signal maybe transferred on at least one signal line in respective signal paths120 a-d. Buffer devices 100 a-d may receive and/or transmit a strobesignal with data on signal paths 120 a-b. In an embodiment, write datais provided to buffer devices 100 a-d on signal paths 120 a-d and astrobe signal is provided on signal paths 120 a-d along side write data.In an embodiment, a strobe signal is provided from buffer devices 100a-d on signal paths 120 a-d along side read data on signal paths 120a-d. In an embodiment, a strobe signal is synchronized with or travelsalong side the write and/or read data. An edge of the strobe signal hasa temporal relationship or is aligned with an edge of a data signalrepresenting write and/or read data.

In an embodiment, addresses (for example, row and/or column addresses)for accessing particular memory locations in a particular integratedcircuit memory device and/or commands are provided on signal path 121from a memory module connector interface. In an embodiment, a commandrelates to a memory operation of a particular integrated circuit memorydevice. For example, a command may include a write command to storewrite data at a particular memory location in a particular integratedcircuit memory device and/or a read command for retrieving read datastored at a particular memory location from a particular integratedcircuit memory device. Also, multiple memory devices in different dataslices can be accessed simultaneously. In embodiments, a command mayinclude row commands, column commands such as read or write, maskinformation, precharge and/or sense command. In an embodiment, controlinformation is transferred on signal path 121 over a common set of linesin the form of a time multiplexed packet where particular fields in thepacket are used for including command operation codes and/or addresses.Likewise, packets of read data may be transferred from integratedcircuit memory devices via buffers 100 a-d on respective signal paths120 a-d to a memory module connector interface. In an embodiment, apacket represents one or more signals asserted at particular bit windows(or a time interval) for asserting a signal on particular signal lines.

In an embodiment, chip select information may be transferred on one ormore signal lines in signal path 121. In an embodiment, chip selectinformation may be one or more chip select signals on respective signallines having predetermined voltage values or states (or logic values)that select and enable operation of a “chip” or integrated circuitmemory device/buffer device.

In embodiments, memory module 100 communicates (via a memory moduleconnector interface) with a master device (e.g., a processor orcontroller).

FIG. 2 illustrates an embodiment of a memory module topology having asplit multi-drop control/address/clock bus. In particular, memory module200 includes a split multi-drop control/address bus 221 coupled tobuffers 100 a-d and a memory module connector interface. With referenceto FIG. 2, a first portion of bus 221 is terminated by termination 230and a second portion of bus 221 is terminated by termination 231. In anembodiment, the impedance of termination 230 matches the impedance ofthe first portion of bus 221 (Z0) coupled to buffers 100 c-d and theimpedance of termination 231 matches the impedance of the second portionof bus 221 (Z1) coupled to buffers 100 a-b. In an embodiment, impedanceZ0 equals impedance Z1. In embodiments, terminations 230 and 231, singlyor in combination, are disposed on memory module 100, buffer devices 100a and 100 d or packages used to house buffer devices 100 a and 100 d.

FIG. 3 illustrates a memory module topology having a single multi-dropcontrol/address/clock bus terminated by termination 330. In anembodiment, the impedance of termination 330 matches the impedance ofsignal path 121 (or control/address/clock bus). In embodiments,termination 330, singly or in combination, is disposed on memory module300 or on buffer device 100 d.

FIG. 4 illustrates a memory module topology that provides data betweeneach integrated circuit buffer device and a memory module connectorinterface. In an embodiment, each signal path 120 a-d is terminated byan associated termination 420 a-d, respectively. In an embodiment,terminations 420 a-d have respective impedances that match the impedanceZ0 of each of the signal paths 120 a-d. In embodiments, terminations 420a-d, singly or in combination, are disposed on memory module 400, eachof buffer devices 100 a-d or packages used to house buffer devices 100a-d.

Referring to FIG. 1, a control/address signal rate ratio of signal path121 to signal path 103 may be 2:1 (or other multiples such as 4:1, 8:1,etc.) so that a memory module connector interface is able to operate asfast as specified while memory devices 101 a-d may operate at half(quarter, eighth, etc) the control/address signaling rate so thatrelatively lower cost memory devices may be used. Similarly, a datasignal rate of one of signal paths 120 a-d to one of signal paths 102a-d may be 2:1 (or other multiple such as 4:1, 8:1, etc) so that amemory module connector interface is able to operate as fast asspecified while memory devices 101 a-d may operate at half (quarter,eighth, etc.) the data signaling rate so that relatively lower costmemory devices may be used.

FIG. 5 illustrates a memory module topology including a plurality ofintegrated circuit memory devices and a plurality of integrated circuitbuffer devices with an integrated circuit buffer device 501 for control,address and/or clock information. Memory module 500 is similar to memorymodule 100 except that buffer device 501 is coupled to signal paths 121and 121 a-b. Buffer device 501 outputs control, address and/or clockinformation to buffer devices 100 a-b on signal path 121 a and to bufferdevices 100 c-d on signal path 121 b. In an embodiment buffer device 501copies control, address and/or clock information received on signal path121 and repeats the control, address and/or clock information on signalpaths 121 a-b. In an embodiment, buffer device 501 is a clocked bufferdevice that provides a temporal relationship with control and addressinformation provided on signal paths 121 a-b. In an embodiment, signalpaths 121 a-b include at least one signal line to provide a clock signaland/or clock information. In an embodiment, buffer device 501 includes aclock circuit 1870 as shown in FIG. 18. In an embodiment, buffer device501 receives control information, such as a packet request, thatspecifies an access to at least one of the integrated circuit memorydevices 101 a-d and outputs a corresponding control signal (on signalpath 121 a and/or 121 b) to the specified integrated circuit memorydevice.

FIG. 6 illustrates a memory module topology similar to that illustratedin FIG. 5 except that a termination 601 is coupled to signal path 121 onmemory module 600. In an embodiment, the impedance of termination 601matches the impedance Z0 of signal path 121. In embodiments, termination601 is disposed on memory module 600 or buffer device 501 or a packageused to house buffer device 501.

FIG. 7 illustrates a memory module topology that provides data to and/orfrom each integrated circuit buffer device and terminations coupled tosignal paths. In an embodiment, each signal path 120 a-d is terminatedby associated terminations 701 a-d, respectively. In an embodiment,terminations 701 a-d have respective impedances that match the impedanceZ0 of each of the signal paths 120 a-d. In embodiments, terminations 701a-d, singly or in combination, are disposed on memory module 700, bufferdevices 100 a-d or packages used to house buffer devices 100 a-d.

FIG. 8 illustrates a memory module topology having a split multi-dropsignal path between a buffer device for control, address and/or clockinformation and the plurality of buffer devices. In particular, memorymodule 800 includes a split multi-drop control/address bus 121 a-bcoupled to buffers 100 a-d and a buffer device 501. In an embodiment, afirst portion of bus 121 a is terminated by termination 801 and a secondportion of bus 121 b is terminated by termination 802. In an embodiment,the impedance of termination 801 matches the impedance of the first leg(Z0) and the impedance of termination 802 matches the impedance of thesecond leg (Z1). In an embodiment, impedance Z0 equals impedance Z1. Inembodiments, terminations 801 and 802, singly or in combination, aredisposed on memory module 800, buffer devices 100 a and 100 d orpackages used to house buffer devices 100 a and 100 d.

Referring to FIG. 5, a control/address signal rate ratio of signal path121 to signal path 121 a (or 121 b) to signal path 103 may be 2:1:1 (orother multiples such as 4:1:1, 8:1:1, etc.) so that other multi-drop bustopology embodiments using signal paths 121 a (or 121 b) and signal path103 do not have to necessarily operate as high a signal rate as anembodiment that uses signal path 121 as shown in FIG. 1. Also like FIG.1, a control/address signal rate ratio of signal path 121 to signal path103 may be 2:1 (or other multiples such as 4:1, 8:1, etc.) so that amemory module connector interface is able to operate as fast asspecified while memory devices 101 a-d may operate at half (or quarter,eighth, etc.) the control/address signaling rate so that relativelylower cost memory devices may be used. Similarly, a data signal rate ofone of signal paths 120 a-d to one of signal paths 102 a-d may be 2:1(or other multiple such as 4:1, 8:1, etc.) so that a memory moduleconnector interface is able to operate as fast as the specifiedsignaling rate while memory devices 101 a-d may operate at half (orquarter, eighth, etc.) the data signaling rate so that relatively lowercost memory devices may be used.

FIG. 9A illustrates a top view of a memory module topology including aplurality of integrated circuit memory devices and a plurality ofintegrated circuit buffer devices coupled to a connector interface. Inan embodiment, memory module 900 includes a substrate 910 having astandard dual in-line memory module (“DIMM”) form factor or other moduleform factor standards, such as small outline DIMM (“SO-DIMM”) and verylow profile DIMM (“VLP-DIMM”). In alternate embodiments, substrate 910may be, but is not limited to, a wafer, printed circuit board (“PCB”),package substrate like BT epoxy, flex, motherboard, daughterboard orbackplane, singly or in combination.

In an embodiment, memory module 900 includes pairs of memory devices 101a-b and buffer devices 100 a-d disposed on a first side of substrate910. In alternate embodiments, more or less memory devices and bufferdevices are used. In an embodiment, pairs of memory devices 101 c-d arealso disposed on a second side of memory module 900 as shown in a sideand bottom view of memory module 900 in FIGS. 9B and 9C. In anembodiment, each memory device and buffer device are housed in separatepackages. In alternate embodiments, memory devices and buffer devicesmay be housed in MCP package embodiments described herein.

Memory module 900 includes connector interface 920 that has differentinterface portions for transferring data and control/address/clocksignals. For example, a first side of memory module 900 includesconnector interface portions 920 a-d used to transfer data signals and aconnector interface portion 930 a used to transfer control/addresssignals. In an embodiment, connector interface portion 930 a alsotransfers a clock signal and/or clock information. In an embodiment, asecond side of memory module 900 including connector interface portions920 e-h are used to transfer data signals and a connector interfaceportion 930 b is used to transfer control/address signals. In anembodiment, connector interface portion 930 b also transfers a clocksignal and/or clock information.

In an embodiment, connector interface 920 is disposed on an edge ofsubstrate 910. In an embodiment, a memory module 900 is inserted into asocket 940 disposed on substrate 950. In an embodiment, substrate 950 isa main board or PCB with signal paths 960 a-b for transferring signalson substrate 950. In an embodiment, signal paths 960 a and 960 b aresignal traces or wires. In an embodiment, signal paths 960 a and 960 bare coupled to other sockets disposed on substrate 950 that may haveanother memory module inserted and/or coupled to a master.

In an embodiment, connector interface portions include at least onecontact or conducting element, such as a metal surface, for inputtingand/or outputting an electrical signal. In alternate embodiments, acontact may be in the form of any one of or a combination of a ball,socket, surface, signal trace, wire, a positively or negatively dopedsemiconductor region and/or pin, singly or in combination. In anembodiment, a connector interface as described herein, such as connectorinterface 920, is not limited to physically separable interfaces where amale connector or interface engages a female connector (or socket 940)or interface. A connector interface also includes any type of physicalinterface or connection, such as an interface used in asystem-in-a-package (“SIP”) where leads, solder balls or connectionsfrom a memory module are soldered to a circuit board.

In an alternate embodiment, memory module 900 is included in an embeddedmemory subsystem, such as one in a computer graphics card, video gameconsole or a printer. In an alternate embodiment, memory module 900 issituated in a personal computer or server.

In an embodiment, a master communicates with memory modules illustratedin FIGS. 1-9 and 16-17. A master may transmit and/or receive signals toand from the memory modules illustrated in FIGS. 1-9 and 16-17. A mastermay be a memory controller, peer device or slave device. In embodiments,a master is a memory controller, which may be an integrated circuitdevice that contains other interfaces or functionality, for example, aNorthbridge chip of a chipset. A master may be integrated on amicroprocessor or a graphics processor unit (“GPU”) or visual processorunit (“VPU”). A master may be implemented as a field programmable gatearray (“FPGA”). Memory modules, signal paths, and a master may beincluded in various systems or subsystems such as personal computers,graphics cards, set-top boxes, cable modems, cell phones, game consoles,digital television sets (for example, high definition television(“HDTV”)), fax machines, cable modems, digital versatile disc (“DVD”)players or network routers.

In an embodiment, a master, memory modules and signal paths are in oneor more integrated monolithic circuits disposed in a common package orseparate packages.

FIG. 10 is a block diagram illustrating an embodiment of a device 1000having a plurality of integrated circuit memory devices 101 a-d and abuffer 100 a. Here, data (read and/or write) may be transferred betweenthe plurality of integrated circuit memory devices 101 a-d and buffer100 a on a signal path 1006 (data). Signal path 1006 is a signal pathsituated internal to device 1000 and corresponds to signal paths 1113a-d and 1114 shown in FIG. 11. Signal path 1006 is a bus for providingbidirectional data signals between a plurality of integrated circuitmemory devices 101 a-d and buffer 100 a. An example of bidirectionaldata signals includes signals traveling from one or more of integratedcircuit memory devices 101 a-d to buffer 100 a and also signalstraveling from buffer 100 a to one or more of integrated circuit memorydevices 101 a-d. Signal path 1005 is a signal path internal to device1000 and corresponds to signal paths 1116 a-d and 1117 shown in FIG. 11.Signal path 1005 is a bus for providing unidirectionalcontrol/address/clock signals from a buffer 100 a to a plurality ofintegrated circuit memory devices 101 a-d. In an example of aunidirectional bus, signals travel in only one direction, i.e., in thiscase, from only buffer 100 a to one or more of integrated circuit memorydevices 101 a-d. Signal path 1005 includes individual control signallines, for example, a row address strobe line, column address strobeline, chip select line, etc., and address signal lines. Signal path 1005may include a fly-by clock line to transfer a clock signal from buffer100 a to integrated circuit memory devices 101 a-d. Signal path 1005 maytransfer a clock signal from one or more integrated circuit memorydevices 101 a-d to buffer 100 a.

In an embodiment, buffer 100 a communicates with a serial presencedetect (“SPD”) device to store and retrieve parameters and configurationinformation regarding device 1000 and/or memory module 900. In anembodiment, an SPD 1002 is a non-volatile storage device. Signal path1004 couples SPD 1002 to buffer 100 a. In an embodiment, signal path1004 is an internal signal path for providing bidirectional signalsbetween SPD 1002 and buffer 100 a.

In an embodiment, SPD 1002 is an EEPROM device. However, other types ofSPD 1002 are possible, including but not limited to a manual jumper orswitch settings, such as pull-up or pull-down resistor networks tied toa particular logic level (high or low), which may change state when amemory module is added or removed from a system.

In an embodiment, SPD 1002 is a memory device that includes registersthat stores configuration information that can be easily changed viasoftware during system operation, allowing a high degree of flexibility,and making configuration operations that are transparent to an end user.

In an embodiment illustrated in FIG. 18, functionality of the SPDmentioned above may be integrated into buffer device 100 a using aregister set, such as configuration register set 1881. Referring to FIG.18, SPD logic and interface 1820 c may be preconfigured with informationpertaining to the buffer and memory devices connected to the buffer, ormay store information pertaining to only one of the memory devices orthe buffer device 100 a. Control inputs to the buffer may determine whena storage node within the register set will sample the information topreload or preconfigure the SPD logic and interface 1820 c. The termregister may apply either to a single-bit-wide register ormulti-bit-wide register.

In an embodiment illustrated by FIG. 10, SPD 1002 stores informationrelating to configuration information of memory module 900 or a memorysystem. For example, configuration information may include repair andredundancy information to repair a defective memory device, defectivememory cells or peripheral circuits on a memory device, and/or signalpath. In an embodiment, SPD configuration information includes memorymodule population topology, such as a number, a position and a type ofmemory device in a package and/or on a memory module, or rank, if any.SPD configuration information may include an amount of memory capacityof one or more memory modules and/or timing information to levelizesignals between memory modules and a master device in a memory system.In an embodiment, SPD configuration information includes a serializationratio for interfaces in a buffer and/or information regardingconfiguring the width of a buffer. In an embodiment, SPD configurationinformation includes a first value that represents the desired width ofbuffer device 100 a or includes multiple values that represent the rangeof possible widths of the buffer device 100 a, and a second value thatrepresents the desired width of interface 1820 b as illustrated in FIG.18.

In an embodiment, SPD configuration information includes timinginformation or parameters for accessing memory devices, such as a timeto access a row or the memory device, a time to access a column of thememory device, a time between a row access and a column access, a timebetween a row access and a precharge operation, a time between a rowsense applied to a first bank of a memory array and a row sense appliedto a second bank of the memory array and/or a time between a prechargeoperation applied to a first bank in a memory array and a prechargeoperation applied to a second bank of the memory array.

In an embodiment, the stored timing information may be expressed interms of time units where a table of values maps specific time units tospecific binary codes. During an initialization or calibration sequence,a master or a buffer may read SPD configuration information anddetermine the proper timing information for one or more memory devices.For example, a master may also read information representing the clockfrequency of a clock signal from an SPD 1002, and divide the retrievedtiming information by a clock period of a clock signal. (The clockperiod of the clock signal is the reciprocal of the clock frequency ofthe clock signal). Any remainder resulting from this division may berounded up to the next whole number of clock cycles of the clock signal.

Signal paths 120 a and 121, as shown in FIG. 10, are coupled to buffer100 a. In an embodiment, signal path 121 transfers unidirectionalcontrol/address/clock signals to buffer 100 a. In an embodiment, signalpath 120 a transfers bidirectional or unidirectional data signals to andfrom buffer 100 a. Other interconnect and external connect topologiesmay also be used for device 1000 in alternate embodiments. For example,buffer 100 a may be coupled to a single multi-drop control bus, a splitmulti-drop control bus, or a segmented multi-drop bus.

In an embodiment, device 1000 has two separate power sources. Powersource V1 supplies power to one or more memory devices (memory devices101 a-d) on memory module 900. Power source V2 supplies power to one ormore buffers (buffer 100 a) on memory module 900. In an embodiment, thebuffer 100 a has internal power regulation circuits to supply power tothe memory devices 101 a-d.

FIG. 11 illustrates a device 1100 including a plurality of integratedcircuit memory dies 1101 a-d and a buffer die 1100 a housed in or upon acommon package 1110 according to embodiments. As described herein inother embodiments and illustrated in FIGS. 12-15 and 35, a plurality ofintegrated circuit memory dies 1101 a-d and buffer die 1100 a aredisposed in multiple package type embodiments. For example, a pluralityof integrated circuit memory dies 1101 a-d and a buffer die 1100 a maybe stacked, on a flexible tape, side-by-side or positioned in separatepackages on a device substrate. Buffer die 1100 a is used to providesignals, including control/address/clock information and data, between aplurality of integrated circuit memory dies 1101 a-d and a deviceinterface 1111 that includes contacts 1104 a-f. In an embodiment, one ormore contacts 1104 a-f is similar to contacts of connector interface920. Contacts 1104 a-f are used to couple device 1100 to substrate 910,and in particular to signal paths 120 a and 121, of memory module 100 inan embodiment. Device interface 1111 also includes signal paths 1118 and1115 to transfer signals between contacts 1104 a-f and buffer 100 a viabuffer interface 1103. Signals are then transferred between a pluralityof memory dies 1101 a-d and buffer die 1100 a via buffer interface 1103and signal paths 1117 (disposed in device interface 1111) and 1116 a-das well as signal paths 1114 (disposed in device interface 1111) and1113 a-d. In an embodiment, spacers 1102 a-c are positioned betweenintegrated circuit memory dies 1101 a-d. In an embodiment, spacers 1102a-c are positioned to dissipate heat. Similarly, buffer die 1100 a isdisposed away from a plurality of integrated circuit memory dies 1101a-d to alleviate heat dissipation near the memory devices. In anembodiment, signal paths are coupled to each other and integratedcircuit memory dies 1101 a-d by a solder ball or solder structure.

FIG. 12 illustrates a stacked package device 1200 having a package 1210containing a plurality of integrated circuit memory dies 1101 a-d and aseparate package 1290 having a buffer die 1100 a. Both packages 1210 and1290 are stacked and housed to make device 1200. In an embodiment, aplurality of integrated circuit memory dies has separate packages and isstacked on package 1290. Device 1200 has similar components illustratedin FIG. 11. Buffer die 1100 a communicates with a plurality ofintegrated circuit memory dies 1101 a-d as described herein. Device 1200has memory dies 1101 a-d stacked upon buffer die 1100 a and separated bycontacts 1201 a-d. In an embodiment, contacts 1201 a-d are solder ballsthat couple signal paths 1117 and 1114 to signal paths 1202 and 1203that are coupled to buffer interface 1103.

FIG. 13 illustrates devices 1300 and 1301 having a plurality ofintegrated circuit memory devices 101 a-b (101 a-c in device 1301) and abuffer device 100 a that are disposed on a flexible tape 1302 accordingto embodiments. Buffer device 100 a communicates with a plurality ofintegrated circuit memory devices as described herein. Signal path 1305disposed on or in flexible tape 1302 transfers signals between aplurality of integrated circuit memory devices 101 a-c and buffer 100 a.Contacts, such as a grid array of balls 1304, couple each integratedcircuit memory device in a plurality of integrated circuit memorydevices 101 a-c and a buffer 100 a to signal path 1305 in flexible tape1302 in an embodiment. Adhesive 1303 may be used to couple a pluralityof integrated circuit memory devices 101 a-c to each other and to abuffer 100 a in an embodiment. Device 1300 and 1301 are disposed incommon package in an embodiment.

FIG. 14 illustrates a device 1400 having a plurality of integratedcircuit memory dies 1101 a-d and 1401 a-d and a buffer die 1100 a thatare disposed side-by-side and housed in a package 1410. Device 1400 hassimilar components illustrated in FIG. 11. Buffer die 1100 acommunicates with a plurality of integrated circuit memory dies 1101 a-dand 1401 a-d as described herein. In an embodiment, a plurality ofintegrated circuit memory dies 1101 a-d and 1401 a-d and a buffer die1100 a are disposed side-by-side on a substrate 1450 that is coupled todevice interface 1411. A plurality of integrated circuit memory dies1401 a-d is separated by spacers 1402 a-c. In an embodiment, a singleintegrated circuit memory die 1101 d and a single integrated circuitmemory die 1401 d are disposed side-by-side with buffer die 1100 a.Device interface 1411 includes contacts 1104 a-f. Signals aretransferred between buffer interface 1103 and contacts 1104 a-f bysignal paths 1418 and 1415. Signals are transferred between bufferinterface 1103 and signal paths 1116 a-d (or integrated circuit memorydies 1101 a-d) by signal path 1417. Similarly, signals are transferredbetween buffer interface 1103 and signal paths 1113 a-d (or integratedcircuit memory dies 1401 a-d) by signal path 1414.

FIG. 15 illustrates a device 1500 having a plurality of integratedcircuit memory dies 1101 a-b and a buffer die 1100 a that are housed inseparate packages 1501, 1505 and 1520, respectively. Device 1500 hassimilar components illustrated in FIG. 11. Buffer die 1100 acommunicates with integrated circuit memory dies 1101 a-b as describedherein. Integrated circuit memory dies 1101 a-b and a buffer die 1100 aare disposed on substrate 1530 that includes signal paths 1504, 1509,1515 and 1518. Integrated circuit memory die 1101 a includes memoryinterface 1507 having contacts 1508. Integrated circuit memory die 1101b includes memory interface 1503 having contacts 1541. Buffer die 1100 aincludes a buffer interface 1103 having contacts 1560. Signals aretransferred between buffer interface 1103 and contacts 1104 a-f bysignal paths 1515 and 1518. Signals are transferred between bufferinterface 1103 and integrated circuit memory die 1101 a by signal path1509 via memory interface 1507 and contacts 1508. Similarly, signals aretransferred between buffer interface 1103 and integrated circuit memorydie 1101 b by signal path 1504 via memory interface 1503 and contacts1541. As described herein, device 1500 is coupled to a memory module 900via contacts 1104 a-f.

FIG. 16 illustrates a memory module having an SPD 1603 according to anembodiment. Memory module 1610 includes a plurality of integratedcircuit memory devices (or dies) and buffer devices (or dies) disposedon substrate 930 along with SPD 1603. FIG. 16 illustrates a memorymodule 1610 having a single SPD 1603 that can be accessed by each bufferdevice 100 a-b positioned on substrate 930. Signal path 1601 allowsaccess to SPD 1603 from connector interface 920 and one or more buffers100 a-b. In an embodiment, signal path 1601 is a bus. SPD 1603 may haveconfiguration and/or parameter information written to or read by amaster by way of connector interface 920 and signal path 1601. Likewise,buffers 100 a-b may write to or read from SPD 1603 via signal path 1601.

FIG. 17 illustrates a memory module 1710 with each device 1711 a-b ordata slice a-b having an associated SPD 1720 a-b, buffer device (or die)100 a-b and at least one integrated circuit memory device 101 a (or die)according to an embodiment. The plurality of buffers 100 a-b andassociated plurality of SPDs 1720 a-b are disposed on substrate 930.Configuration and/or parameter information is accessed from SPDs 1720a-b using signal path 1701, which is coupled, to connector interface 920and each SPD 1720 a-b. In particular, signal path 1701 couples SPD 1720a-b of device 1711 a-b to connector interface 920. In an embodiment,signal path 1701 is a bus. In an alternate embodiment, signal path 1701couples SPD 1720 a and SPD 1720 b in a daisy chain or serial topology.In an embodiment, one or more buffer devices 100 a-b of devices 1711 a-bmay access (read and/or write) respective SPDs 1720 a-b. Likewise, amaster may access (read and/or write) respective SPDs 1720 a-b usingsignal path 1701. In an embodiment, configuration and/or parameterinformation is transferred using a header field or other identifier sothat SPDs coupled in a daisy chain may forward the SPD information tothe intended destination SPD.

FIG. 18 illustrates a block diagram of a buffer device 100 a (or die,such as buffer die 1100 a) according to embodiments. Buffer 100 aincludes buffer interface 1103 a, interfaces 1820 a-c, redundancy andrepair circuit 1883, multiplexer 1830, request and address logic circuit1840, data cache and tags circuit 1860, computations circuit 1865,configuration register set 1881, and clock circuit 1870, singly or incombination.

In a memory read operation embodiment, buffer 100 a receives controlinformation (including address information) that may be in a packetformat from a master on signal path 121 and in response, transmitscorresponding signals to one or more, or all of memory devices 101 a-don one or more signal paths 1005. One or more of memory devices 101 a-dmay respond by transmitting data to buffer 100 a which receives the datavia one or more signal paths 1006 and in response, transmitscorresponding signals to a master (or other buffer). A master transmitsthe control information via one or more signal paths 121 and receivesthe data via one or more signal paths 120 a.

By bundling control and address information in packets, protocolsrequired to communicate to memory devices 101 a-d are independent of thephysical control/address interface implementation.

In a memory write operation embodiment, buffer 100 a receives controlinformation (including address information) that may be in a packetformat from a master on signal path 121 and receives the write data forone or more memory devices 101 a-d that may be in a packet format from amaster on signal path 120 a. Buffer 100 a then transmits correspondingsignals to one or more, or all of memory devices 101 a-d on one or moresignal paths 1006 so that the write data may be stored.

A master transmits the control/address/clock information via one or moresignal paths 121 and transmits the write data via one or more signalpaths 120 a.

In an embodiment, simultaneous write and/or read operations may occurfor different memory devices in memory devices 101 a-d.

In an embodiment, control information that is provided to buffer 100 acauses one or more memory operations (such as write and/or readoperations) of one or more memory devices 100 a-d, while the samecontrol information may be provided to buffer 100 b which causes thesame memory operations of one or more memory devices 100 a-d associatedwith buffer 100 b. In another embodiment, the same control informationmay be provided to buffer 100 a and buffer 100 b, yet different memoryoperations occur for the one or more memory devices 100 a-d associatedwith each buffer 100 a-b.

In an embodiment, buffer interface 1103 a couples signal paths 121 and120 a to buffer 100 a as shown in FIG. 10. In an embodiment, bufferinterface 1103 a corresponds to buffer interface 1103 shown in FIGS. 11,12, 14 and 15. In an embodiment, buffer interface 1103 a includes atleast one transceiver 1875 (i.e. transmit and receive circuit) coupledto signal path 120 a to transmit and receive data and at least onereceiver circuit 1892 coupled to signal path 121 to receivecontrol/address/clock information. In an embodiment, signal paths 121and 120 a include point-to-point links. Buffer interface 1103 a includesa port having at least one transceiver 1875 that connects to apoint-to-point link. In an embodiment, a point-to-point link comprisesone or a plurality of signal lines, each signal line having no more thantwo transceiver connection points. One of the two transceiver connectionpoints is included on buffer interface 1103 a. Buffer interface 1103 amay include additional ports to couple additional point-to-point linksbetween buffer 100 a and other buffer devices on other devices and/ormemory modules. These additional ports may be employed to expand memorycapacity as is described in more detail below. Buffer 100 a may functionas a transceiver between a point-to-point link and other point-to-pointlinks. In an embodiment, buffer interface 1103 a includes a repeatercircuit 1899 to repeat data, control information and/or a clock signal.In an embodiment, buffer interface 1103 a includes a bypass circuit 1898to transfer signals between connector interface portions.

In an embodiment, termination 1880 is disposed on buffer 100 a and isconnected to transceiver 1875 and signal path 120 a. In this embodiment,transceiver 1875 includes an output driver and a receiver. Termination1880 may dissipate signal energy reflected (i.e., a voltage reflection)from transceiver 1875. Termination 1880, as well as other terminationdescribed herein, may be a resistor or capacitor or inductor, singly ora series/parallel combination thereof. In alternate embodiments,termination 1880 may be external to buffer 100 a. For example,termination 1880 may be disposed on a substrate 910 of a memory module900 or on a package used to house buffer 100 a.

Interface 1820 a includes at least one transmitter circuit 1893 coupledto signal path 1005 to transmit control/address/clock information to oneor more memory devices. In an embodiment, interface 1820 a includes atransceiver that may transfer control/address/clock information betweenbuffers disposed on a common memory module or different memory modules.

Interface 1820 b includes a transceiver 1894 coupled to signal path 1006to transfer data between buffer 100 a and one or more memory devices 101a-d as illustrated in FIG. 10. SPD logic and interface 1820 c includes atransceiver 1896 coupled to signal path 1004 to transfer configurationand/or parameter information between buffer 100 a and an SPD 1002 asillustrated in FIG. 10. In an embodiment, interface 1820 c is used totransfer configuration and/or parameter information as illustrated inFIGS. 16 and 17.

According to an embodiment, multiplexer 1830 may performbandwidth-concentrating operations between buffer interface 1103 a andinterface 1820 b as well as route data from an appropriate source (i.e.target a subset of data from memory devices, internal data, cache orwrite buffer). The concept of bandwidth concentration involves combiningthe (smaller) bandwidth of each data path coupled to a memory device ina multiple data signal path embodiment to match the (higher) overallbandwidth utilized by buffer interface 1103 a. In an embodiment,multiplexing and demultiplexing of throughput between the multiplesignal paths that may be coupled to interface 1820 b and bufferinterface 1103 a is used. In an embodiment, buffer 101 a utilizes thecombined bandwidth of multiple data paths coupled to interface 1820 b tomatch the bandwidth of interface buffer interface 1103 a.

In an embodiment, data cache and tags circuit 1860 (or cache 1860) mayimprove memory access time by providing storage of most frequentlyreferenced data and associated tag addresses with lower access latencycharacteristics than those of the plurality of memory devices. In anembodiment, cache 1860 includes a write buffer that may improveinterfacing efficiency by utilizing available data transport windowsover an external signal path to receive write data and address/maskinformation. Once received, this information is temporarily stored in awrite buffer until it is ready to be transferred to at least one memorydevice over interface 1820 b.

Computations circuit 1865 may include a processor or controller unit, acompression/decompression engine, etc., to further enhance theperformance and/or functionality of buffer 100 a. In an embodiment,computations circuit 1865 controls the transfer of control/address/clockinformation and data between buffer interface 1103 a and interfaces 1820a-c.

Clock circuit 1870 may include a clock generator circuit (e.g., DirectRambus® Clock Generator), which may be incorporated onto buffer 101 aand thus may eliminate the need for a separate clock generating device.

In an alternate embodiment, clock circuit 1870 include clock alignmentcircuits for phase or delay adjusting an internal clock signal withrespect to an external clock signal, such as a phase lock loop (“PLL”)circuit or delay lock loop (“DLL”) circuit. Clock alignment circuits mayutilize an external clock from an existing clock generator, or aninternal clock generator to provide an internal clock, to generateinternal synchronizing clock signals having a predetermined temporalrelationship with received and transmitted data and/or controlinformation.

In an embodiment, clock circuit 1870 receives a first clock signalhaving a first frequency via signal path 121 and generates a secondclock signal (via interface 1820 a) to memory device 101 a using thefirst clock signal and also generates a third clock signal (viainterface 1820 a) to memory device 101 b using the first clock signal.In an embodiment, the second and third clock signals have apredetermined temporal (phase or delay) relationship with the firstclock signal.

In an embodiment, a transmit circuit (such as in transceivers 1875, 1896and 1894 shown in FIG. 18) transmits a differential signal that includesencoded clock information and a receiver circuit (such as in transceiver1875, 1896 and 1894) receives a differential signal that includesencoded clock information. In this embodiment, a clock and data recoverycircuit (such as clock circuit 1870) is included to extract the clockinformation encoded with the data received by the receiver circuit.Likewise, clock information may be encoded with data transmitted by thetransmit circuit. For example, clock information may be encoded onto adata signal, by ensuring that a minimum number of signal transitionsoccur in a given number of data bits.

In an embodiment, a transceiver 1875 transmits and receives a first typeof signal (for example, a signal having specified voltage levels andtiming), while transceivers 1894 (and/or transmit circuit 1893)transmits and receives a second different type of signal. For example,transceiver 1875 may transmit and receive signals for a DDR2 memorydevice and transceivers 1894 may transmit and receive signals for a DDR3memory device.

In an embodiment, the control information and/or data that is providedto buffer 100 a (by way of signal paths 121 and 120) may be in adifferent protocol format or have different protocol features than thecontrol information and/or data provided to one or more memory devices100 a-d from buffer 100 a. Logic (for example computation circuit 1865)in buffer 100 a performs this protocol translation between the controlinformation and/or data received and transmitted. A combination of thedifferent electrical/signaling and control/data protocol constitute aninterface standard in an embodiment. Buffer 100 a can function as atranslator between different interface standards—one for the memorymodule interface (for example connector interface 920) and another forone or more memory devices 100 a-d. For example, one memory moduleinterface standard may require reading a particular register in aparticular memory device disposed on the memory module. Yet, a memorymodule may be populated with memory devices that do not include theregister required by the memory module interface standard. In anembodiment, buffer 100 a may emulate the register required by the memorymodule interface standard and thus allow for the use of memory devices100 a-d that operate under a different interface standard. This bufferfunctionality, combined with the module topology and architecture,enables a memory module to be socket compatible with one interfacestandard, while using memory devices with a different interfacestandard.

In an embodiment, buffer 100 a includes a redundancy and repair circuit1883 to test and repair the functionality of memory cells, rows or banksof a memory device, entire memory devices (or periphery circuits) and/orsignal paths between buffer 100 a and memory devices 101 a-d. In anembodiment, redundancy and repair circuit 1883 periodically, during acalibration operation and/or during initialization, tests one or more ofmemory devices 101 a-d by writing a predetermined plurality of values toa storage location in a selected memory device (for example, usingtransceiver 1894 and a look-up table storing the predetermined values)using a selected data path and then reading back the storedpredetermined plurality of values from the selected memory device usingthe selected data path. In an embodiment, when the values read from thestorage location of the selected memory device do not match the valueswritten to the storage location, redundancy and repair circuit 1883eliminates access by buffer 100 a to the selected memory device and/orselected signal path. In an embodiment, a different signal path to adifferent memory device may be selected and this testing function may beperformed again. If selecting the different signal path results in anaccurate comparison of read predetermined values to the predeterminedvalues in redundancy and repair circuit 1883 (or a pass of the test),the different memory address to a different memory location, within orto another memory device, is selected or mapped thereafter. Accordingly,future write and/or read operations to the defective memory locationwill not occur.

In an embodiment, any multiplexed combination of control information(including address information) and data intended for memory devices 101a-d coupled with buffer 100 a is received via buffer interface 1103 a,which may, for example extract the address and control information fromthe data. For example, control information and address information maybe decoded and separated from multiplexed data on signal path 120 a andprovided on signal path 1895 to request and address logic circuit 1840from buffer interface 1103 a. The data may then be provided toconfigurable serialization/deserialization circuit 1891. Request andaddress logic circuit 1840 generates one or more control signals totransmitter circuit 1893.

Interfaces 1820 a and 1820 b include programmable features inembodiments. A number of control signal lines and/or data signal linesbetween buffer 100 a and memory devices 101 a-d are programmable inorder to accommodate different numbers of memory devices. Thus, morededicated control signal lines are available with an increased number ofmemory devices. Using programmable dedicated control lines and/or datalines avoids any possible load issues that may occur when using a bus totransfer control signals between memory devices and a buffer 100 a. Inanother embodiment, additional data strobe signals for each byte of eachmemory device may be programmed at interface 1820 b to accommodatedifferent types of memory devices, such as legacy memory devices thatrequire such a signal. In still a further embodiment, interfaces 1820 aand 1820 b are programmable to access different memory device widths.For example, interfaces 1820 a and 1820 b may be programmed to connectto 16 “×4” width memory devices, 8 “×8” width memory devices or 4 “×16”width memory devices. Likewise, buffer interface 1103 a has aprogrammable width for signal path 120 a.

Configurable serialization/deserialization circuit 1891 performsserialization and deserialization functions depending upon a storedserialization ratio. As a memory device access width is reduced from itsmaximum value, memory device access granularity (measured in quanta ofdata) is commensurately reduced, and an access interleaving ormultiplexing scheme may be employed to ensure that all storage locationswithin memory devices 101 a-d can be accessed. The number of signalpaths 1006 may be increased or decreased as the memory device accesswidth changes. Signal path 1006 may be subdivided into severaladdressable subsets. The address of the transaction will determine whichtarget subset of signal path 1006 will be utilized for the data transferportion of the transaction. In addition, the number of transceiver,transmitter and/or receiver circuits included in interfaces 1820 a and1820 b that are employed to communicate with one or more memory devices101 a-d may be configured based on the desired serialization ratio.Typically, configuration of the transceivers may be effectuated byenabling or disabling how many transceivers are active in a giventransfer between one or more memory devices 101 a-d and buffer interface1103 a. In an embodiment, a data rate of transferring data at bufferinterface 1103 a is a multiple or ratio of a data rate of transferringdata on one or more signal paths 1006 coupled to memory devices 101 a-d.

Buffer 100 a provides a high degree of system flexibility. New interfacestandards of memory devices may be phased in to operate with a master ora memory system that supports older interface standards by modifyingbuffer 100 a. In an embodiment, a memory module may be inserted using anolder memory module interface or socket, while newer generation memorydevices may be disposed on the memory module. Backward compatibilitywith existing generations of memory devices may be preserved. Similarly,new generations of masters, or controllers, may be phased in whichexploit features of new generations of memory devices while retainingbackward compatibility with existing generations of memory devices.Similarly, different types of memory devices that have different costs,power requirements and access times may be included in a single commonpackage for specific applications.

FIG. 19 illustrates an integrated circuit memory device 1900 (or amemory die) in an embodiment. Integrated circuit memory device 1900corresponds to one or more integrated circuit memory devices 101 a-d inembodiments. Integrated circuit memory device 1900 includes a memorycore 1900 b and a memory interface 1900 a. Signal paths 1950 a-b, 1951a-b, 1952 and 1953 are coupled to memory interface 1900 a. Signal paths1950 a-b transfer read and write data. Signal paths 1951 a-b transferaddress information, such as a row address and a column address inpackets, respectively. Signal path 1952 transfers control information.Signal path 1953 transfers one or more clock signals. In an embodiment,signal paths 1950 a-b correspond to signal path 120 a shown in FIG. 10and signal paths 1951 a-b, 1952 and 1953 correspond to signal path 121in FIG. 10.

Memory interface 1900 a includes at least one transmitter and/orreceiver for transferring signals between memory device 1900 and signalpaths 1950 a-b, 1951 a-b, 1952 and 1953. Write demultiplexer (“demux”)1920 and read multiplexer (“mux”) 1922 are coupled to signal path 1950a, while write demux 1921 and read mux 1923 are coupled to signal path1950 b. Write demux 1920-21 provide write data from signal paths 1950a-b to memory core 1900 b (in particular sense amplifiers 0-2a and0-2b). Read mux 1922-23 provide read data from memory core 1900 b tosignal paths 1950 a-b (in particular sense amplifiers Na and Nb).

Demux and row packet decoder 1910 is coupled to signal path 1951 a andDemux and column packet decoder 1913 is coupled to signal path 1951 b.Demux and row packet decoder 1910 decodes a packet and provides a rowaddress to row decoder 1914. Demux and Column packet decoder 1913provides a column address and mask information to column and maskdecoder 1915.

Control registers 1911 are coupled to signal path 1952 and providecontrol signals to row decoder 1914 and column and mask decoder 1915 inresponse to register values.

A clock circuit is coupled to signal path 1953 to provide a transmitclock signal TCLK and a receive clock signal RCLK in response to one ormore clock signals transferred on signal path 1953. In an embodiment,write demux 1920 and 1921 provide write data from signal paths 1950 a-bto memory core 1900 b in response to an edge of receive clock signalRCLK. In an embodiment, read mux 1922 and 1923 provide read data frommemory core 1900 b to signal paths 1950 a-b in response to an edge of atransmit clock signal TCLK. In an embodiment, the clock circuitgenerates a clock signal on signal path 1953 (to a buffer device) thathas a temporal relationship with read data that are output on signalpaths 1950 a-b.

Row decoder 1914 and column and mask decoder 1915 provide controlsignals to memory core 1900 b. For example, data stored in a pluralityof storage cells in a memory bank is sensed using sense amplifiers inresponse to a row command. A row to be sensed is identified by a rowaddress provided to row decoder 1914 from demux and row packet decoder1910. A subset of the data sensed by a sense amplifier is selected inresponse to a column address (and possible mask information) provided bydemux and column packet decoder 1913.

A memory bank in memory banks 0-N of memory core 1900 b includes amemory array having a two dimensional array of storage cells. Inembodiments, memory banks 0-N include storage cells that may be DRAMcells, SRAM cells, FLASH cells, ferroelectric RAM (“FRAM”) cells,magnetoresistive or magnetic RAM (“MRAM”) cells, or other equivalenttypes of memory storage cells. In an embodiment, integrated circuitmemory device 1900 is a DDR integrated circuit memory device or latergeneration memory device (e.g., DDR2 or DDR3). In an alternateembodiment, integrated circuit memory device 1900 is an XDR™ DRAMintegrated circuit memory device or Direct Rambus® DRAM (“DRDRAM”)memory device. In an embodiment, integrated circuit memory device 1900includes different types of memory devices having different types ofstorage cells housed in a common package.

FIGS. 20A-B illustrate signal paths between memory module interfaceportions and a plurality of integrated circuit buffer devices. Inparticular, FIG. 20A illustrates how each buffer device 100 a-d hassignal paths for data signals coupled to each connector interfaceportion 920 a-h. In an embodiment, FIGS. 20A-B illustrate signal pathsbetween buffer devices and connector interfaces of memory module 900that include a plurality of memory devices as shown in FIGS. 9A-C. Forexample, FIG. 20B which shows an expanded section of FIG. 20A,illustrates how data signal paths 2003 and 2004 provide data signalsbetween connector interface portions 920 a and 920 e and buffer device100 a. FIG. 20A also illustrates how signal paths for control/addresssignals, such as control/address signal paths 2001 and 2002, coupleconnector interface portions 930 a and 930 b to buffer devices 100 a-d.In an embodiment, each signal path 2001 and 2002 is a multi-drop bus asshown in FIG. 1.

FIGS. 21A-D illustrate memory system point-to-point topologies includinga master 2101 and at least one memory module having a plurality ofintegrated circuit memory devices (The plurality of memory devices onrespective memory modules are not illustrated in FIGS. 21A-D, 22A-C,23A-C and 24A-B for clarity). In an embodiment, FIGS. 21A-D, 22A-C,23A-C and 24A-B illustrate signal paths between memory modules, such asmemory module 900 as shown in FIGS. 9A-C, and other memory modulesand/or masters. FIGS. 21A-D illustrate expanding memory capacity andbandwidth as well as different configurations. In particular, master2101 is coupled to interfaces (such as sockets) 2102 and 2103 by signalpaths 2120, 2121 a-b, 2122 and 2123 in Dynamic Point-to-Point (“DPP”)system 2100 a. In an embodiment, master 2101, interfaces 2102 and 2103as well as signal paths 2120, 2121 a-b, 2122 and 2123 are disposed on asubstrate, such as a printed circuit board (“PCB”). In an embodiment,memory modules may be inserted and/or removed (unpopulated) frominterfaces 2102 and 2103. In an embodiment, signal paths 2120, 2121 a-b,2122 and 2123 are signal traces on a PCB. In an embodiment, signal paths2120 and 2121 a-b provide data between data signal paths on a memorymodule, such as signal paths 120 a and 120 b shown in FIG. 1, and master2101. In an embodiment, signal paths 2122 and 2123 providecontrol/address information to the memory modules (via interfaces 2102and 2103 and in particular connector interface portions 930 b of thememory modules) from master 2101. In particular, control/addressinformation is provided from signal paths 2122 and 2123 to a signal pathon the memory modules, such as signal path 121 shown in FIG. 1.

FIG. 21A illustrates a DPP system 2100 a that simultaneously accessestwo buffer devices in memory modules coupled to interfaces 2102 and2103. In response to control and address information provided on signalpaths 2122 and 2123 from master 2101, the two buffers 101 a output datasimultaneously from connector interface portions 920 a and 920 e,respectively, onto signal paths 2120 and 2121 a, that are coupled tomaster 2101. In an embodiment, signal paths 2120 and 2121 a arepoint-to-point links. In an embodiment, a point-to-point link includesone or a plurality of signal lines, each signal line generally havingtwo transceiver connection points, each transceiver connection pointcoupled to a transmitter circuit, receiver circuit or transceivercircuit. For example, a point-to-point link may include a transmittercircuit coupled at or near one end and a receiver circuit coupled at ornear the other end. The point-to-point link may be synonymous andinterchangeable with a point-to-point connection or a point-to-pointcoupling.

In an embodiment, the number of transceiver points along a signal linemay distinguish between a point-to-point link and a bus. For example, apoint-to-point link generally includes only two transceiver connectionpoints while a bus generally includes more than two transceiver points.In some instances a point to point link can be mixed with bussed signallines, where the bussed single lines may be used to provide sidebandfunctionality such as maintenance, initialization or test.

Several embodiments of point-to-point links include a plurality of linktopologies, signaling, clocking and signal path types. Embodimentshaving different link architectures include simultaneous bi-directionallinks, time-multiplexed bi-directional links and multiple unidirectionallinks. Voltage or current mode signaling may be employed in any of theselink topologies.

FIG. 21B illustrates a DPP with Continuity Module system 2100 b foraccessing a buffer device 101 a in a memory module coupled to interface2103 while a continuity memory module 2105 is coupled to interface 2102.In an embodiment, master 2101 outputs a single set of control/addressinformation on signal paths 2122 and 2123. Data is output from connectorinterfaces 920 a and 920 e of the memory module coupled to interface2103 in response to the single set of control/address information. Datais provided to master 2101 on signal path 2120 via signal path 2121 band a bypass circuit in continuity memory module 2105. The bypasscircuit passes the data from connector interface portion 920 e toconnector interface portion 920 a in continuity memory module 2105. Datais also provided to master 2101 by signal path 2121 a.

FIG. 21C illustrates a DPP bypass system 2100 c similar to system 2100 bexcept that a buffer device 101 a (rather than continuity memory module2105) in a memory module includes a bypass circuit for passing the datafrom connector interface portion 920 e to connector interface portion920 a of the memory module inserted in interface 2102.

FIG. 21D illustrates a DPP bypass system 2100 d similar to system 2100 cexcept that data is accessed from buffer device 101 a of the memorymodule coupled to interface 2102 and buffer device 101 a of the memorymodule coupled to interface 2103 includes a bypass circuit for passingthe data from connector interface portion 920 a to connector interfaceportion 920 e.

In an embodiment, a clock signal or clock information is provided onsignal paths 2122 and 2123, on a separate signal path from a clocksource or master 2101, or along the data signal paths 2121 a-b.

FIGS. 22A-C illustrate memory system daisy chain topologies including amaster 2101 and at least one memory module having a plurality ofintegrated circuit memory devices. In particular, FIGS. 22A-C illustratehow half of the bandwidth, as compared to system 2100 a-d, is obtainedwhen accessing a single memory module in an embodiment. FIG. 22Aillustrates a Daisy Chain system 2200 a that includes a buffer 101 a ina memory module coupled to interface 2103 that provides data (by way ofconnector interface portion 920 e) on signal path 2121 a in response toa single set of control/address information output by master 2101 ontosignal paths 2122 and 2123. No module is coupled to interface 2102.

FIG. 22B illustrates a Daisy Chain system 2200 b that is similar tosystem 2200 a except a memory module is coupled to interface 2102.

FIG. 22C illustrates a Daisy Chain system 2200 c similar to system 2200b except that data accessed from a buffer device 101 a in a memorymodule is coupled to interface 2102 rather than interface 2103. Bufferdevice 101 a in a memory module coupled to interface 2103 provides abypass circuit to allow data to be received at interface portion 920 aand output at interface portion 920 e of the memory module coupled tointerface 2103. Data is thus passed from data path 2121 b to data path2121 a and ultimately to master 2101.

FIGS. 23A-C and 24A-B illustrate memory system topologies including amaster to provide control/address information to a plurality ofintegrated circuit buffer devices. In particular, FIG. 23A illustrates aDedicated/Fly-by system 2300 a that includes a master 2101 that providescontrol/address information to memory modules 2301 a and 2301 b (inparticular to integrated circuit buffer devices 101 a-d on each memorymodule) by signal paths 2311 and 2310, respectively. In an embodiment,signal paths 2310 and 2311 are separate and carry control/addressinformation for each respective memory module. In an embodiment, signalpath 2311 does not pass through or include a signal path in memorymodule 2301 b. In an embodiment, signal path 2311 does not pass throughor include an interface, such as a socket, used for memory module 2301b. The double headed arrow in FIGS. 23A-C, 24A-B and 25A-B illustratethe data information (read and write data) transferred on separate datapaths between memory modules 2301 a-b (and in particular from bufferdevices) and master 2101. In an embodiment, a clock signal or clockinformation is provided on signal paths 2310 and 2311, on a separatesignal path from a clock source or master 2101, or along the data signalpaths.

Signal path 2311 is terminated by termination 2350 a and signal path2310 is terminated by termination 2350 b. In an embodiment, theimpedance of termination 2350 a matches the impedance of a portion ofthe signal path 2311 (multi-drop bus 2320 a) on memory module 2310 a,(Z0) and the impedance of termination 2350 b approximately matches theimpedance of a portion of the signal path 2310 (multi-drip bus 2320 b)on memory module 2301 b (Z1). In an embodiment, impedance Z0approximately equals impedance Z1. In embodiments, terminations 2350 aand 2350 b, singly or in combination, are disposed on memory module,buffer devices or packages used to house buffer devices. FIG. 23Billustrates a Stub/Fly-by system 2300 b similar to system 2300 a exceptthat a single signal path 2320 provides control/address information frommaster 2101 to memory modules 2301 a and 2301 b (in particular tointegrated circuit buffer devices 101 a-d on each memory module). In anembodiment, memory modules 2301 a and 2301 b include stubs/internalsignal paths (multi-drop bus) 2320 a-b coupled to a single common signalpath 2320 that are disposed on memory modules 2301 a-b. In anembodiment, a portion of signal path 2320 passes through or includes aninterface, such as a socket, used for memory module 2301 b. Memorymodules 2301 a and 2301 b are terminated similar to system 2300 a.

FIG. 23C illustrates a Serpentine system 2300 c similar to system 2300 aexcept that a single signal path 2320 provides control/addressinformation from master 2101 to memory modules 2301 a and 2301 b (inparticular to integrated circuit buffer devices 101 a-d on each memorymodule) without using stubs on respective memory modules as illustratedin FIG. 23B. In an embodiment, a single signal path 2330 couples master2101 to memory modules 2301 a and 2301 b. In an embodiment signal path2330 includes a first external signal path portion between master 2101and memory module 2301 b; a second signal path portion disposed on thememory module 2301 b and coupled to the first signal path portion aswell as to respective buffer devices 101 a-d; a third external signalpath portion 2331 coupled to the second signal path portion and alsocoupled to memory module 2301 a; and a fourth signal path portiondisposed on the memory module 2301 a and coupled to the third signalpath portion 2331 as well as to respective buffer devices 101 a-d onmemory module 2301 a. Termination 2350 a, in an embodiment, is notdisposed on memory module 2301 a in order to ensure that memory modulesare interchangeable. Termination 2350 a may be disposed on a PCB orelsewhere in a system.

FIG. 24A illustrates a Dedicated/Tree system 2400 a similar to system2300 a except that memory modules 2401 a-b include buffer devices 101a-d that are coupled by way of a tree structure/topology signal path2413. A tree structure/topology may also be referred to as a “forked,”“T” or “hybrid T” topology. In particular, memory module 2401 a iscoupled to signal path 2311 by signal path 2413 a disposed on memorymodule 2401 a that then branches in to signal paths 2413 b and 2413 c.Signal path 2413 b then is coupled to buffer devices 101 a and 101 b bybranches or signal paths 2413 d and 2413 e. Signal path 2413 c,likewise, is coupled to buffer devices 101 c and 101 d by branches orsignal paths 2413 f and 2413 g. In an embodiment, memory module 2401 bhas a similar tree structure signal path 2413 to couple buffer devices101 a-d to signal path 2310.

FIG. 24B illustrates a Stub/Tree system 2400 b similar to system 2400 ashown in FIG. 24A that includes tree structure signal path 2413 inmemory modules 2401 a-b. System 2400 b illustrates signal path 2320including stubs/signal paths 2320 a and 2320 b that couple master 2101to memory modules 2401 a and 2401 b, respectively. Stub/signal path 2320a is coupled to signal path 2413 a disposed on memory module 2401 a andstub/signal path 2320 b is coupled to signal path 2413 a disposed onmemory module 2401 b.

In embodiments, termination may be disposed on buffers 101 a-d, memorymodules 2401 a-b and/or elsewhere in a system, such as on a PCB.

FIGS. 25A-B illustrate memory modules having different memory capacityor different sized address spaces. In particular, memory module addressspace 2501 on a first memory module is larger than memory module addressspace 2502 on a second memory module. In an embodiment, memory moduleaddress space 2501 is twice as large as memory module address space2502. For example, memory module address space 2501 may store 2 gigabyte(GB) of information and memory module address space 2502 may store 1 GBof information. Increasing the number or density of integrated circuitmemory devices disposed on a memory module may increase address space.

FIG. 25A illustrates how half (or portion) of the available signal pathwidth, for example half of a bus width, is used to access the first halfof memory module address space 2501 (overlapping address space) whilethe other half of the available signal path width is used to accessmemory module address space 2502.

FIG. 25B illustrates how a larger capacity memory module is able to usea full signal path by accessing a first half (or portion) of theavailable signal path width coupled directly to the larger capacitymemory module and by way of accessing a second half (or portion) of theavailable signal path width coupled to the smaller capacity memorymodule using bypassing through the smaller capacity memory module. FIGS.26-29 illustrate how non-overlap address space of a larger memory modulemay be accessed in various embodiments.

FIGS. 26A-B illustrate a system 2600 to access different sized/capacity(address space) memory modules during different modes of operation, afirst mode of operation and a second mode of operation (or bypass mode).System 2600 includes a master 2101 coupled to memory module 2601 bysignal path 2610 and memory module 2602 by signal path 2612. Memorymodules 2601 and 2602 are coupled by signal path 2611. In an embodiment,memory modules 2601 and 2602 represent memory modules includingintegrated circuit memory devices and buffer devices as describedherein. In an embodiment, memory module 2601 has a larger address spacethan memory module 2602. In an embodiment, signal paths 2610-2612 arepoint-to-point links that provide read/write data. In embodiments,control/address/clock information is provided on separate signal pathsas described herein. Memory modules 2601 and 2602 may include bypasscircuits 2630 a-b.

In a first mode of operation (or a non-bypass mode) illustrated in FIG.26A, read data 2601 a (stored in an overlapping address space) isprovided on signal path 2610 to master 2101 from memory module 2601 inresponse to control/address information provided by master 2101 tomemory module 2601. Similarly, read data 2602 a (stored in anoverlapping address space) is provided on signal path 2612 to master2101 from memory module 2602 in response to control/address informationprovided by master 2101 to memory module 2602. In the first mode ofoperation, signal path 2611 is not used.

In a second mode of operation (or a bypass mode) illustrated in FIG.26B, read data 2601 b (stored in a non-overlapping address space ofmemory module 2601) is provided on signal path 2610 to master 2101 frommemory module 2601 in response to control/address information providedby master 2101 to memory module 2601. Read data 2601 c (stored in anon-overlapping address space of memory module 2601) is provided onsignal path 2611 to memory module 2602 in response to control/addressinformation provided by master 2101 to memory module 2601. Bypasscircuit 2630 b then provides read data 2601 c to signal path 2612 andeventually to master 2101.

Write data from master 2101 may be provided to memory modules 2601 and2602 similar to how read data is obtained during a first and second modeof operation.

In embodiments, modes of operation are determined in response to acontrol signal from master 2101, or other circuit or in response toreading configuration information stored in a separate storage circuitin a device, such as an SPD device or register on the buffer orcontroller device, disposed on system 2600. Modes of operation may bedetermined at initialization, periodically or during calibration ofsystem 2600.

In embodiments, bypass circuits 2630 a-b (as well as bypass circuits2630 c-d shown in FIG. 27) correspond to bypass circuit 2900 asdescribed below and shown in FIG. 29 and/or bypass circuit 1898 shown inFIG. 18. In embodiments, these bypass circuits can be incorporated onthe buffer devices on the module.

FIG. 27 illustrates a system 2700 including master 2101 coupled to atleast four memory modules 2701-2704 by way of interfaces 2701 a-d. In anembodiment, interfaces 2701 a-d are female sockets disposed on asubstrate, such as a backplane, motherboard or PCB, to receive male edgeinterfaces of memory modules 2701-2704. In an embodiment, memory modules2701-2704 represent memory modules including integrated circuit memorydevices and buffer devices as described herein as well as at least oneof bypass circuits 2630 a-d.

Master 2101 is coupled to memory module 2701 by signal path 2710. Signalpath 2711 couples memory module 2701 to memory module 2704. In anembodiment, bypass circuit 2630 a allows read and write data to betransferred between signal paths 2711 and 2710 either to or from masterdevice 2101 in response to control/address information provided tomemory module 2704.

Master 2101 is coupled to memory module 2702 by signal path 2712. Signalpath 2713 couples memory module 2702 to memory module 2703. Signal path2714 couples memory module 2703 to memory module 2704. In an embodiment,bypass circuits 2630 b and 2630 c allow read and write data to betransferred between signal paths 2712 and 2713, as well as signal paths2713 and 2714, either to or from master device 2101 in response tocontrol/address information provided to memory modules 2702-04.

Master 2101 is coupled to memory module 2703 by signal path 2714. Signalpath 2716 couples memory module 2703 to memory module 2704. In anembodiment, bypass circuit 2630 c allows read and write data to betransferred between signal paths 2714 and 2716 either to or from masterdevice 2101 in response to control/address information provided tomemory modules 2703-04.

Master 2101 is coupled to memory module 2704 by signal path 2717. In anembodiment, read and write data is transferred on signal path 2717 to orfrom master device 2101 in response to control/address informationprovided to memory module 2704.

FIGS. 28A-B illustrate a system 2700 to access different capacity/sized(address space) memory modules during different modes of operation thatis similar in operation to that of system 2600. FIG. 28A illustratesaccessing data in a first mode of operation, such as accessing read datafrom different sized memory modules that may be disposed in interfaces2701 a-d. Table 2810 illustrates how different sized memory modules maybe disposed in respective interfaces 2701 a-d during a first mode ofoperation. For example, interfaces 2701 a-d may be coupled to all“small” sized memory modules as indicated by the first row of Table2810. Alternatively, interface 2701 a may be coupled to a “large” sizedmemory module; interface 2701 b may be coupled to a “small” sized memorymodule; interface 2701 c may be coupled to a “large” sized memorymodule; and interface 2701 d may be coupled to a “small” sized memorymodule, as indicated by the second from last row of Table 2810.

In a first mode of operation (non-bypass mode) as illustrated by FIG.28A, data 2810 a is provided on signal path 2717; data 2820 a isprovided on signal path 2714; data 2830 is provided on signal path 2712;and data 2840 is provided on signal path 2710.

Table 2820 illustrates how different sized memory modules may bedisposed in respective interfaces 2701 a-d during a second mode ofoperation (bypass mode). For example, interfaces 2701 c-d may be coupledto “small” sized memory modules and interfaces 2701 a-b include bypasscircuits 2802 and 2801 as indicated by the first row of Table 2820.Alternatively, interface 2701 c may be coupled to a “large” sized memorymodule; and interface 2701 d may be coupled to a “small” sized memorymodule. Interfaces 2701 a-b include bypass circuits 2802 and 2801, asindicated by Table 2820.

In a second mode of operation (bypass mode) as illustrated by FIG. 28B,read data 2810 b is provided on signal path 2717 and read data 2810 c isprovided on signal paths 2711 and 2710 (via bypass circuit 2802). Readdata 2820 b is provided on signal path 2714 and read data 2820 c isprovided on signal paths 2713 and 2712 (via bypass circuit 2801).

In embodiments, bypass circuits 2801 and/or 2802 are disposed in acontinuity module, integrated circuit buffer device, interface (forexample a socket) and/or memory module. In an embodiment, bypasscircuits 2801 and 2802 are conductive elements, such as metal traces orwires that may be disposed manually on an interface or memory module. Inan embodiment, bypass circuits 2801 and 2802 correspond to bypasscircuit 2900 shown in FIG. 29.

FIG. 29 illustrates a bypass circuit 2900 used in a write operationaccording to an embodiment. Bypass circuit 2900 includes receiver andtransmitter circuits 2901 a-e and 2902 a-d coupled to a signal pathincluding signal paths DQ[0:3] and RQ. In an embodiment, bypass circuit2900 is included in an integrated circuit buffer device, such ascorresponding to bypass circuit 1898 in buffer interface 1103 a,disposed on a memory module and/or corresponding to bypass circuits 2630a-d shown in FIGS. 26A-B and 27. For example, signal paths DQ[0:1] arecoupled to connector interface portion 920 a and signal paths DQ[2:3]are coupled to connector interface portion 920 b as shown in FIGS.20A-B. In an embodiment, signal paths DQ[0:1] are coupled to an adjacentmaster or memory module and signal paths DQ[2:3] are coupled to a memorymodule in a memory system.

Receiver circuits 2901 a-d receive write data signals from signal pathsDQ[0:3] and provide write data to data width translator circuit 2950and/or back out to a signal path by way of transmitters 2902 a-d andbypass elements 2905-2910. Receiver circuit 2901 e receives writeaddress signals from signal path RQ and provides write addresses to datawidth translator circuit 2950. Receiver circuit 2901 a is coupled tobypass elements 2906 and 2908 to reroute received data signals totransmitter circuits 2902 b and 2902 c in response to control signals(not shown) provided to bypass elements 2906 and 2908. Receiver circuit2901 b is coupled to bypass elements 2905 and 2910 to reroute receiveddata signals to transmitter circuits 2902 a and 2902 d in response tocontrol signals (not shown) provided to bypass elements 2905 and 2910.Receiver circuit 2901 c is coupled to bypass element 2907 to reroutereceived data signals to transmitter circuit 2902 a in response tocontrol signals (not shown) provided to bypass element 2907. Receivercircuit 2901 d is coupled to bypass element 2909 to reroute receiveddata signals to transmitter circuit 2902 b in response to controlsignals (not shown) provided to bypass element 2909.

As can be seen, write data may be rerouted from a single signal path DQ0to another single signal path DQ1. Write data may be also rerouted fromtwo signal paths DQ0 and DQ1 to signal paths DQ2 and DQ3.

In an embodiment, bypass elements 2905-2910 function independently asrespective switches to allow a signal (represented by a voltage level)to be passed from a receiver circuit to a transmitter circuit. In anembodiment, bypass elements 2905-2910 are semiconductors such asnegative and/or positive-channel metal-oxide (NMOS/PMOS) semiconductorswith a control signal (such as a voltage) provided to a gate of thesemiconductor while a source and/or a drain is coupled to a transmitterand/or receiver circuit. In an alternate embodiment, other types ofsemiconductors or switches may be used. In an embodiment, controlsignals (not shown) provided to bypass elements 2905-2910 are providedby master 2101 or from a programmable register, such as an SPD device.In an embodiment, control signals are provided by a master after readingmemory capacity information of memory modules stored in one or more SPDdevices. In an embodiment, control signals provided to bypass elementsmay be provided in response to a manual jumper, programmable fuse orregister. In an embodiment, control signals provided to bypass elementsmay be provided by one or more integrated circuit buffer devices inresponse to one or more integrated circuit buffer devices reading areceived address/control information. For example, when an address isreceived that identifies a memory location that is not provided on aparticular memory module (non-overlapping address space or smallercapacity memory module), control signals are provided to bypass elementsfrom the integrated circuit buffer device that received theaddress/control information (in a bypass mode) to enable data to bererouted from the larger capacity memory module to another destination,such as a master.

In an embodiment, bypass elements 2905-2910 may be disposed before orleft of receiver and transmitter circuits 2901 a-d and 2902 a-d as wellas in or after (right of) data width translator circuit 2950 (forexample, after a clock barrier or boundary). Bypass elements 2905-2910may be disposed in a master, an interface (such as a socket) and/or amemory module (outside of a buffer device). Bypass elements 2905-2910may also be disposed internal to an integrated circuit buffer, asopposed to an interface of an integrated circuit buffer device, or in anintegrated circuit memory device.

In an embodiment, rerouted write data may be resynchronized by atransmitter circuit using a different or the same clock signal that isused by the receiver circuit in receiving the read data. Also, writedata that has been rerouted by bypass elements may be transmitted in afast analog mode.

Stored read data from integrated circuit memory devices disposed on amemory module are provided on signal paths DQ_DRV[0:3] by way of anintegrated circuit buffer device. Read data is levelized or delays areprovided to the read data by a selector circuit, such as multiplexers(mux) 2903 a-d, and delay circuits 2904 a-d in response to DELAY[0:3]control signals. Signal paths DQ_DRV[0:3] are input to delay circuits2904 a-d and a first input (“0 input”) of mux 2903 a-d, while an outputof delay circuits 2904 a-d is provided to a second input (“1 input”) ofmux 2903 a-d. DELAY[0:3] control signals select an output of mux 2903a-d or whether a delay is introduced into read data on signal pathsDQ_DRV[0:3]. In an embodiment, delay circuits 2904 a-d may introduce aprogrammable delay in response to a control signal (not shown). Controlsignals provided to delay circuits 2904 a-d as well as DELAY[0:3]control signals may be provided similar to control signals provided tobypass elements 2905-2910 as described above.

In an embodiment, delay circuits 2904 a-d are inverters, registersand/or a series of inverters and/or registers that may introduceprogrammable delay to a read signal on signal paths DQ_DRV[0:3]. Theamount of delay provided to read data by delay circuits 2904 a-d may belonger than the amount of time for providing read data to delay circuits2904 a-d, or longer than a data cycle time.

In an embodiment, multiplexers 2903 a-d and delay circuits 2904 a-d maybe disposed before or left of receiver and transmitter circuits 2901 a-dand 2902 a-d. For example, multiplexers 2903 a-d and delay circuits 2904a-d may be disposed in a master, interface (such as a socket) and/ormemory module. In an embodiment, multiplexers 2903 a-d and delaycircuits 2904 a-d may be disposed in data width translator circuit 2950and/or left of data width translator circuit 2950. For example,multiplexers 2903 a-d and delay circuits 2904 a-d may be disposedinternal to an integrated circuit buffer, as opposed to an interface ofan integrated circuit buffer device, or in an integrated circuit memorydevice.

Levelization or the amount of delay (if any) provided to read data onsignal paths DQ_DRV[0:3] is dependent upon the signal path (between amemory module and a master) used by a system to provide the read data tothe master (or flight time or amount of time to transfer read data froma memory module to a master and/or another memory module). For examplein a system 2600 shown in FIG. 26B, delay is introduced into data 2601 bso that data 2601 b arrives at master 2101 at the approximate same timedata 2601 c arrives at master 2101 because data 2601 c travels a longerpath (as compared to data 2601 b) on signal paths 2611 and 2612 as wellas through memory module 2602 (or at least through an integrated bufferdevice/interface of memory module 2602).

Data width translator circuit 2950 may be configurable to translate dataof various widths into data suitable for a fixed-width memory die ordevice disposed on a memory module. Data width translator circuit 2950,in accordance with some embodiments, uses a data-mask signal toselectively prevent memory accesses to subsets of physical addresses.This data masking divides physical address locations of the memory dieinto two or more temporal subsets of the physical address locations,effectively increasing the number of uniquely addressable locations in aparticular memory die. As used herein, the term “width” refers to thenumber of bits employed to represent data.

A data width translator circuit 2950 allows memory modules, such asmemory modules 2601 and 2602, to vary the effective width of theirexternal memory module interfaces without varying the width of theinternal memory device/die interfaces. A memory system thus may supporta first mode of operation and a second mode of operation (bypass mode).In the bypass mode of operation, memory module 2601 uses both signalpath 2610 and signal paths 2611 and 2612 (via memory module 2602).

In accordance with an embodiment, data width translator circuit 2950 cantranslate data of width one, two, or four on signal paths DQ[0:3] intofour-bit-wide data on signal path IDQ[0:3]. Address translator circuit2970 translates address signals on signal path RQ to signal path IRQwhich is coupled to one or more memory devices. This flexibility allowsone or a combination of memory modules to be used in an extensiblepoint-to-point memory topology. Similarly, data width translator circuit2950 can translate data of width one, two, or four on signal pathsIDQ[0:3] into four-bit-wide data on signal path DQ[0:3].

Data width translator circuit 2950 includes a data translator circuit2960, an address translator circuit 2970, and a DLL 2980. DLL 2980produces an internal differential clock signal ICLK locked (or having atemporal relationship) to a like-identified incoming differential clocksignal CLK, typically from an associated master or a clock-generatordevice. Though not shown, a memory device disposed on a memory modulemay receive the same or a similar clock signal CLK from data widthtranslator circuit 2950 or a master. Data translator circuit 2960 andaddress translator circuit 2970, responsive to a configuration signalCFG, translate the data on one, two, or four of data signal pathsDQ[0:3] into four-bit-wide data on signal paths IDQ[0:3] for writecycles; and conversely translate four-bit-wide data on signal pathsIDQ[0:3] into one, two, or four-bit-wide data on one or more of externalsignal paths DQ[0:3] for read cycles. In one embodiment, plugging asecond memory module into a two-connector mother board automaticallyasserts configuration signal CFG, causing each of two memory modules toconfigure themselves as half-width (e.g., two bits instead of four)modules. In other embodiments, configuration signal CFG comes from aregister on a memory module (e.g., within data width translator circuit2950) that is addressable by a master and is set, such as via the BIOS,at boot time. In other embodiments, a configuration signal CFG isprovided after reading values stored in a SPD device. In general, anexternal memory module interface conveys data signals of data-width N,an internal memory device interface conveys signals of data-width M, andconfiguration signal CFG is indicative of the ratio of N to M. Someembodiments use a PLL instead of DLL 2980.

A fixed-width memory device disposed on a memory module may include amask line/signal path or pin that can be used in support ofpartial-write operations. For example, double data rate “DDR” memory dieinclude a data-mask pin DM and single data rate “SDR” memory die includea data-mask pin DQM. Memory modules detailed herein may employ data-maskfunctionality to create variable-width modules using fixed-width memorydevices. In an embodiment, a data-mask signal DM is output from datatranslator circuit 2960 to one or more memory devices in order tosynchronize write operations. FIGS. 30A-B, described below, illustrate awrite operation using data width translator circuit 2950 in anembodiment.

In an embodiment, bypass circuit 2900 includes bypass elements 2905-2910and not multiplexers 2903 a-d and delay circuits 2904 a-d. In analternate embodiment, bypass circuit 2900 includes multiplexers 2903 a-dand delay circuits 2904 a-d and not bypass elements 2905-2910. Forexample, memory module 2601 shown in FIG. 26B, and in particular bypasscircuit 2630 a, may include multiplexers 2903 a-d and delay circuits2904 a-d to provide a delay to data 2601 a and not bypass elements2905-2910. Conversely, memory module 2602, and in particular bypasscircuit 2630 b, may include bypass elements 2905-2910 to reroute data2601 c but not multiplexers 2903 a-d and delay circuits 2904 a-d toprovide a delay. In an embodiment, bypass circuit 2900 is disposed in amemory system that does not include an integrated circuit buffer device.

FIGS. 30A-B illustrate a pair of timing charts 3000 and 3001 depictingthe operation of a memory system, or memory module, using data widthtranslator circuit 2950 in a first mode of operation and a second modeof operation (bypass mode). Data to be written to a common address A ina single memory device disposed on a memory module may be transmittedover external signal paths DQ[0:3] as four eight-symbol bursts (a singleeight-symbol burst 0A-0H on signal path DQ0 is shown in FIG. 30B) and anaddress A on signal path RQ. For example, signal path DQ0 conveys eightbinary symbols 0A through 0H for storage at physical address location Ain a fixed width memory device on the memory module. In embodiments, thethree remaining signal paths DQ[1:3] likewise may convey eight symbolsfor storage at address location A. When all signal paths DQ[0:3] areused, the total number of symbols to be stored at a given address A maybe thirty-two (four times eight). Data width translator circuit 2950 mayconvey the thirty-two symbols and corresponding address A to a memorydevice via signal paths IDQ[0:3] and IRQ. The burst length can be longeror shorter in other embodiments.

In an embodiment, data width translator circuit 2950 uses mask signal DMto divide the addressed physical locations in a fixed-width memorydevice into subsets of memory locations addressed separately in the timedomain, a process that may be referred to as “time slicing.” Forexample, a most significant bit(s) (MSB(s)), or any other bits inaddress A, causes data translator circuit 2960 (via a signal fromaddress translator circuit 2970 to data translator circuit 2960) toassert a mask signal DM (DM=1) to block writes to a first set oflocations having address A, and then de-asserts mask signal DM (DM=0) toallow writes to the second set of locations having address A. Thisprocess then may repeat.

FIG. 30A illustrates how data provided from two external signal pathsDQ[0:1] is output on signal paths IDQ[0:3] by data width translatorcircuit 2950 in a bypass mode of operation (i.e. memory modules 2701 and2702 are bypassed as illustrated in FIGS. 27 and 28B). In an embodiment,signal path DQ0 is included in signal path 2717 and signal path DQ1 isincluded in signal path 2711. Data 0A-0H is provided on signal path 2717from master 2101 while data 1A-1H is also provided by master 2101 onsignal path 2711 via memory module 2701 and signal path 2710.

In an embodiment, the address space in memory module 2704 (i.e. memorydevices) is bisected in the time domain. One of the external addressbits of address A is employed to assert mask signal DM every other timeslot. In this embodiment, the MSB of the external address A is zero, somask signal DM is deasserted for every time slot MSB=0 to allow writesduring those time slots.

FIG. 30B illustrates how data provided from an external signal path DQ0(or signal paths DQ[0:3]) is output on signal paths IDQ[0:3] by datawidth translator circuit 2950 in a non-bypass mode of operation (i.e.data is provided to each of the memory modules/sockets as illustrated inFIGS. 27 and 28A). In an embodiment, signal path DQ0 is included insignal path 2717. Data 0A-0H is provided on signal path 2717 from master2101. Similarly, other data may be provided from master 2101 to memorymodules 2701-2703 on signal paths DQ1, DQ2 and DQ3 that are included insignal paths 2710, 2712 and 2714.

FIG. 31 illustrates a method 3100 to adjust read and write data delaysin a system including memory modules having different capacity and abypass circuit. In embodiments, logic blocks illustrated in FIGS. 31 and40 are carried out by hardware, software or a combination thereof. Inembodiments, logic blocks illustrated in FIGS. 31 and 40 illustrateactions or steps. In embodiments, the circuits and/or systemsillustrated herein, singly or in combination, carry out the logic blocksillustrated in FIGS. 31 and 40. Other logic blocks that are not shownmay be included in various embodiments. Similarly, logic blocks that areshown may be excluded in various embodiments. Also, while methods 3100and 4000 shown in FIGS. 31 and 40 are described in sequential logicblocks, steps or logic blocks of methods 3100 and 4000 are completedvery quickly or almost instantaneously.

Method 3100 begins at logic block 3101 where a determination is madewhether to levelize or adjust delays to read and write data in a memorysystem. In an embodiment, this determination may be made atinitialization, periodically or during calibration (testing). Iflevelization is not desired, method 3100 ends. Otherwise, integratedcircuit buffer devices are set to a typical or first mode of operationas illustrated by logic block 3102. In an embodiment, a control signalfrom a master, such as master 2101 shown in FIG. 26A-B, generates acontrol signal to memory modules, and in particular to integratedcircuit buffer devices of the memory modules to operate in a first modeof operation which includes providing read and write data on separatesignal paths (signal paths 2610 and 2612) to or from a master asillustrated in FIG. 26A. In the first mode of operation, no additionaldelay is provided to read and write data, as compared to the second modeof operation described below.

Logic block 3103 illustrates levelizing read data or providing delays toread data to take into account different flight times or distances theread data must travel on different signal paths in reaching a master.For example, signal path 2612 has a longer signal path than signal path2610. Therefore, in order for read data 2601 a and 2602 a from bothmemory modules 2601 and 2602 to reach master 2101 at the approximatesame time, a delay should be introduced into the read data 2601 a toaccount for the longer flight time or distance of signal path 2612. Inan embodiment, delays are provided in response to delay values stored inregisters on the integrated circuit memory devices and programmed by themaster. In alternate embodiments, delays corresponding to respectivememory modules are provided and programmed in the master. Test symbolsor test data may be written and read from the integrated circuit memorydevices to determine the programming of the delay values.

A determination is then made whether a memory system includes differentcapacity memory modules as illustrated by logic block 3104. If differentcapacity memory modules are not present, control transitions to logicblock 3107. Otherwise, control transitions to logic block 3105. In anembodiment, the determination illustrated by logic block 3104 may becompleted by a master reading configuration information of a systemstored in an SPD.

Integrated circuit buffer devices are then set to a second mode ofoperation (bypass mode) as illustrated in logic block 3105. In anembodiment, the bypass mode of operation is set by providing controlsignals to a bypass circuit in an integrated circuit buffer device, forexample bypass elements 2905-2910 in a bypass circuit 2900 asillustrated in FIG. 29.

Read data from a larger capacity memory module is then levelized asillustrated by logic block 3106. For example, delays are added to readdata 2601 b of memory module 2601 (larger capacity) as illustrated inFIG. 26B. In an embodiment, Delay[0:3] control signals are provided tomultiplexers 2903 a-d to select additional delay to data signal onsignal path DQ_DRV[0:3] of bypass circuit 2900 shown in FIG. 29. Thedelay provided in logic block 3106 is in addition to any delay providedin logic block 3103.

Integrated circuit buffers in a smaller capacity memory module are setto a first mode of operation (or a non-bypass mode) as illustrated bylogic block 3109. For example, memory module 2602 in FIG. 26A has anintegrated circuit buffer device that is set to a typical mode ofoperation.

Read data levelization for the smaller capacity memory module is thenperformed as illustrated by logic block 3108.

Write data levelization for data written to memory modules is performedin logic block 3107.

A determination is then made whether a memory system includes differentcapacity memory modules as illustrated by logic block 3110. If differentcapacity memory modules are not present, method 3100 ends. Otherwise,control transitions to logic block 3111. In an embodiment, thedetermination illustrated by logic block 3110 may be completed by amaster reading configuration information of a system stored in a SPD.

Integrated circuit buffer devices are then set to a second mode ofoperation (bypass mode) as illustrated in logic block 3111. In anembodiment, the bypass mode of operation is set by providing controlsignals to a bypass circuit in an integrated circuit buffer device, forexample bypass elements 2905-2910 in a bypass circuit 2900 asillustrated in FIG. 29.

Write data to larger capacity memory modules is then levelized (inaddition to the write data levelization illustrated in logic block 3107)as illustrated by logic block 3112. In an embodiment, additional writedelays are added, in response to stored write delay values, to the writedata at a master, integrated circuit buffer device and/or memory device.Delays to write data may be selected based on whether write data istransferred through a memory module having an integrated circuit bufferdevice in a bypass mode of operation. For example, write data providedto memory module 2601 on signal path 2610 from master 2101 may bedelayed compared to write data provided to memory module 2601 on signalpaths 2612 and 2611 (by way of bypass circuit 2630 b) from master 2101so that the write data may arrive at approximately the same time.

FIGS. 32A-E, 33A-B, 34 and 35 illustrate at least a portion of memorysystem topologies including an integrated circuit buffer device 3201 toprovide control/address information (RQ) to a plurality of integratedcircuit memory devices 101 a-d as well as transferring data (DQ) betweenthe integrated circuit buffer device 3201 and the plurality ofintegrated circuit memory devices 101 a-d. While each of FIGS. 32A-E,33A-B, 34 and 35 illustrate one or more signal paths to transfer eithercontrol/address information (RQ) or data (DQ), other topologies orsignal paths in other Figures may be combined and used to transfercontrol/address information (RQ) and/or data (DQ). For example, FIG. 33Aillustrates a fly-by topology having signal paths 3310 and 3310 a-d thatmay be used for transferring control/address information (RQ); whiledata (DQ) may be transferred using a point-to-point (or segmented)topology or signal paths 3410-3413 shown in FIG. 34. Numerous othertopology combinations may likewise be used in embodiments.

While topologies are illustrated with memory modules 3200 a-e, 3300 a-band 3400, these illustrated topologies in FIGS. 32A-E, 33A-B and 34 maybe used without a memory module. For example, topologies illustrated inFIGS. 32A-E, 33A-B and 34 may be used in an MCP or SIP embodiment. FIG.35 illustrates a particular topology in MCP device 3500.

In embodiments, a master, such as master 2101 may providecontrol/address information and data to one or more integrated circuitbuffer devices 3201 in a topology illustrated in FIGS. 32A-E, 33A-B and34. In an embodiment, a clock signal or clock information is provided onsignal paths from buffer device 3201 illustrated in FIGS. 32A-E, 33A-Band 34, or on a separate signal path from a clock source, master, bufferdevice, or along the data signal paths.

In embodiments, termination may be disposed on buffer 3201, memorymodules 3200 a-e, 3300 a-b and 3400, signal paths, memory devices 101a-d and/or elsewhere in a system, such as on an PCB or substrate. Inembodiments, termination for the signal paths in the topologies shown inFIGS. 32A-E, 33A-B and 34 may be similarly disposed as shown in FIGS.2-4, 6-8 and 23A-C. For example, termination 420 a-d shown in FIG. 4 maybe similarly coupled to signal paths 3410-3413 shown in FIG. 34.

FIGS. 32A-E illustrate forked (data and control/address information)topologies between an integrated circuit buffer device 3201 and aplurality of integrated circuit memory devices 101 a-d. With respect toFIG. 32A, buffer device 3201 is coupled to signal path 3210 disposed onmemory module 3200 a that then branches into signal paths 3210 a and3210 d. Signal path 3210 a then is coupled to memory devices 101 a and101 b by branches or signal paths 3210 b and 3210 c. Signal path 3210 d,likewise, is coupled to memory devices 101 c and 101 d by branches orsignal paths 3210 e and 3210 f.

FIG. 32B illustrates a forked topology similar to the topologyillustrated in FIG. 32A. Signal path 3220 branches into signal paths3220 a and 3220 b that couple memory devices 101 a-b to buffer device3201. Similarly, signal path 3230 branches into signal paths 3230 a and3230 b that couple memory devices 101 c-d to buffer device 3201.

FIG. 32C illustrates a forked/multi-drop bus topology. Buffer device3201 is coupled to signal path 3240 (or a stub) that branches intosignal paths 3240 a and 3240 b (or a bus) that are coupled to signalpaths (or stubs) 3240 c-f coupled to memory devices 101 a-d. Othermemory devices may be coupled to signal paths 3240 a-b.

FIG. 32D illustrates a star topology. Signal path 3250 branches intosignal path 3250 a-d from a common node that couples memory devices 101a-d to buffer device 3201.

FIG. 32E illustrates a forked topology similar to the topologyillustrated in FIG. 32B. Signal path 3260 branches into signal paths3260 a and 3260 b that couple memory devices 101 a-b to buffer device3201.

FIGS. 33A-B illustrate fly-by topologies (data and/or control/addressinformation) between an integrated circuit buffer device 3201 and aplurality of integrated circuit memory devices 101 a-d. FIG. 33Aillustrates a stub/fly-by topology including a buffer device 3201coupled to a signal path 3310 that is coupled to signal paths (stubs)3310 a-d that are coupled to memory devices 101 a-d. FIG. 33Billustrates a split/stub/fly-by topology. A buffer device 3201 iscoupled to a signal path 3320 that is coupled to signal paths (stubs)3320 a-b that are coupled to memory devices 101 a-b. The buffer device3201 is also coupled to a signal path 3330 that is coupled to signalpaths (stubs) 3330 a-b that are coupled to memory devices 101 c-d.Split/stub/fly-by topologies may be divided/split into even furthersections in embodiments.

FIG. 34 illustrates point-to-point (also known as segmented) topology(data and/or control/address information) between an integrated circuitbuffer device 3201 and a plurality of integrated circuit memory devices101 a-d. Separate or segmented signal paths 3410-3413 (in particularpoint-to-point links) couple buffer device 3201 to memory devices 101a-d. A segmented topology for data using separate point-to-point linksis also illustrated in FIGS. 38-39 described below.

FIG. 35 illustrates an MCP (or SIP) topology (data and/orcontrol/address information) between an integrated circuit buffer die1100 a and a plurality of integrated circuit memory dies 1101 a-c.Device 3500 includes a plurality of integrated circuit memory dies 1101a-c and a buffer die 1100 a housed in or upon a common package 3510according to embodiments. A plurality of signal paths 3501 a-c arecoupled to a signal path 3502 that provides data between the integratedcircuit buffer die 1100 a and the plurality of integrated circuit memorydies 1101 a-c. Similarly, a plurality of signal paths 3503 a-c arecoupled to a signal path 3504 that provides control/address informationfrom the integrated circuit buffer die 1100 a to the plurality ofintegrated circuit memory dies 1101 a-c. As described above, a pluralityof integrated circuit memory dies 1101 a-d and buffer die 1100 a may bedisposed with or without spacers and in multiple package typeembodiments.

FIG. 36 is a block diagram of an integrated circuit buffer device 3600(or a buffer die). Buffer device 3600, includes among other circuitcomponents, interfaces 3601 and 3611, register set 3605, data path 3606,data path router 3610, command decode 3607 and address translation 3608.Buffer device 3600 also includes phase locked loop (“PLL”) 3602, JointTest Action Group or IEEE 1149.1 standard (“JTAG”) interface 3603,Inter-IC (“I2C”) interface 3604, pattern generator 3609 and internalmemory array 3612 circuit components.

In a memory read operation, buffer device 3600 operates similar tobuffer 100 a shown in FIG. 18. Buffer device 3600 receives controlinformation (including address information) that may be in a packetformat from a master on signal path 121 and in response, transmitscorresponding signals to one or more, or all of memory devices 101 a-don one or more signal paths 1005. In an embodiment, command decode 3607and address translation 3608 output control signals to data path 3606,data path router 3610 and interface 3611 so that received read memorycommands and received read addresses are decoded and translated tocorresponding control/address signals output on signal path 1005. One ormore of memory devices 101 a-d may respond by transmitting read data tobuffer device 3600 which receives the read data via one or more signalpaths 1006 and in response, transmits corresponding signals to a master(or other buffer). In an embodiment, data path 3606 and data path router3610 (in response to control signals) merge separate read data from morethan one memory device into a single merged read data or read streamoutput at interface 3601.

In an embodiment, memory devices 101 a-d are configured into memoryranks having segmented (point-to-point) signal paths 1006 and a sharedfly-by bus signal path 1005 as illustrated in FIGS. 33A, 34, 38 and 39.A timing chart 3701 shown in FIG. 37B, and described in detail below,illustrates an operation of buffer device 3600 that may increasebandwidth by reducing a time bubble when buffer device 3600 is coupledto ranked memory by segmented signal paths as described below.

In a memory write operation embodiment, buffer 3600 operates similar tobuffer 100 a. Buffer 3600 receives control information (includingaddress information) that may be in a packet format from a master onsignal path 121 and receives the write data for one or more memorydevices 101 a-d that may be in a packet format from a master on signalpath 120 a. In an embodiment, command decode 3607 and addresstranslation 3608 output control signals to data path 3606, data pathrouter 3610 and interface 3611 so that received write memory commandsand received write addresses are decoded and translated to correspondingcontrol/address signals output on signal path 1005. Buffer 3600 thentransmits corresponding signals to one or more, or all of memory devices101 a-d on one or more signal paths 1006 so that the write data may bestored. In an embodiment, data path 3606 and data path router 3610 (inresponse to control signals) segments or parses received write data intotwo or more write portions and directs the write portions to theappropriate signal paths 1006 (via interface 3611) so that the writeportions will be stored in two or more memory devices. Accordingly,buffer 3600 may receive write data having an associated write address toa particular memory device and parses/segments the received write datainto a plurality of different write data portions which are then routedto a plurality of different memory devices at a plurality of differentwrite addresses for storage.

Interfaces 3601 and 3611 correspond to portions of interfaces 1103 a andinterfaces 1820 a-b shown in FIG. 18. For example, interface 3601 mayinclude one or more of transceiver 1875 and receiver circuit 1892 aswell as termination 1880. Interface 3611 may include one or more oftransceiver 1894 and transmitter circuit 1893. In an embodiment,interface 3611 includes circuits to interface with DDR3 memory devicesand interface 3601 includes circuits to interface with DDR2 memorydevices or other type of memory device.

In an embodiment, interface 3611 can be segmented into at least threedifferent configurations or segmentation modes: 1) Four 4-bit interfaces(4×4), 2) Two 4-bit interfaces (2×4) or 3) Two 8-bit interfaces (2×8).The different configurations allow flexibility in memory module ormemory stack configurations. Accordingly, buffer 3600 may interface withhigh-capacity or lower-capacity entry level memory modules or inparticular memory devices. A four 4-bit interface may be used in highcapacity memory modules. A two 8-bit interface may be used for low-costmemory modules. A two 4-bit interface may be used for low-cost memorymodules that still support ECC.

The assignment of strobe pins to data pin groupings is adjusteddepending upon the segmentation mode:

4×4 segmentation mode:

-   -   DQS[0]->DQ[3:0]    -   DQS[1]->DQ[7:4]    -   DQS[2]->DQ[11:8]    -   DQS[3]->DQ[15:12]

2×4 segmentation mode:

-   -   DQS[0]->DQ[3:0]    -   DQS[1]->DQ[7:4]    -   DQS[3:2], DQ[15:8] disabled

2×8 segmentation mode:

-   -   DQS[0]->DQ[7:0]    -   DQS[1]->DQ[15:8]    -   DQS[3:2] disabled

Interface 3601 enters segmentation modes in response to bit valuesstored in register set 3605 and/or one or more control signals fromaddress translation 3608.

Data path router 3610 routes read and write data between data path 3606and interface 3611. Control signals from command decode 3607 and addresstranslation 3608 determine the routing of read/write data. Data pathrouter also receives signals from pattern generator 3609 and internalmemory array 3612. In a mode of operation that emulates operation with amemory device, all memory transactions are routed to and from internalmemory array 3612 rather than interface 3611. Interface 3611 may bedisabled during this mode of operation. In an embodiment, patterngenerator 3609 is used as an alternate source of data (or test patternof data) as well as a source for injecting ECC errors in modes ofoperation. The test pattern of data may be transmitted on eitherinterface 3601 or interface 3611 or some portion of both simultaneously.Similarly, pattern generator 3609 may insert ECC errors on eitherinterface 3601 or interface 3611 or some portion of both simultaneously.In an embodiment, data path router 3610 includes XOR logic used for ECCerror injection. In embodiments, read and write data may proceed throughdata path 3606 in both directions simultaneously. Modes of operation ofbuffer 3600 may be entered by setting one or more bit values inmulti-bit register set (or storage circuit) 3605.

Data path router 3610 includes a write data router 3610 a and read datarouter 3610 b. In an embodiment, write data router 3610 a outputs writedata in response to a WCLK clock signal while the read data router 3610b outputs read data in response to a RCLK clock signal (either thepositive or negative edge of RCLK clock signal). The use of two clockdomains may enable the buffer 3600 to reduce latency and/or operate at ahigher data rate.

During a typical mode of operation, write data router 3610 a receiveswrite data and mask information from data path 3606 and then routes thewrite data (or portion of the write data) to one of four signal paths1006 coupled to interface 3611. Similarly during a read operation, readdata is received from one of four signal paths 1006 coupled to interface3611 and routed to data path 3606.

Data path router 3610 includes a plurality of signal paths used to mergeread data from different memory devices as well as parse write data intowrite data portions to be stored in multiple memory devices.

Command decode 3607 includes a decoder to output control signals to datapath 3606, address translation 3608 and data path router 3610 inresponse to control information received by interface 3601 from signalpath 121. In embodiments, the control information may include memorytransaction commands, such as read or write commands. Other controlinformation may include a command to activate a particular memory bankin a particular memory device or access information having a particularpage size. In an embodiment, command decode 3607 may remap/translate areceived bank address to a different bank address of one or more memorydevices coupled to signal paths 1006.

Address translation circuit 3608 receives an address associated with aparticular memory transaction command by way of signal path 121 andinterface 3601. For example, address translation circuit 3608 receivesan address for reading data associated with a read command for aparticular memory device in a particular memory organization (forexample, number of ranks, number of memory devices, number of banks permemory device, page size, bandwidth). Address translation 3608 thenoutputs control signals (or a translated address and/or control signals)to interface 3611 (and signal path 1005) so that the read data may beread from different memory devices (via signal paths 1006) because thememory organization coupled to interface 3611 is different thanindicated in the read command. In an embodiment, address translation3608 may include a storage circuit to store a look-up table fortranslating addresses. Similarly, write addresses associated with awrite command are received by address translation 3608 which outputscontrol signals (translated write addresses) to interface 3611 andsignal path 1006 so that the corresponding write data from data path3606 may be written to one or more translated write addresses of memorydevices coupled to signal paths 1006.

In an embodiment, information in a received row address field is used tooutput chip select signals. Buffer device 3600 outputs chip selectinformation, such as chip select signals, from interface 3611 to one ormore integrated circuit memory devices in response to information in arow address field received at interface 3601. One or more row addressbit values may be remapped to chip select signals. For example, valuesof two particular row address bits may be used to generate four one-hotchip select signals from interface 3611 to four or more integratedcircuit memory devices.

In an embodiment, information in a received row address field andreceived chip select signals are used to output chip select signals.Buffer device 3600 receives chip select information, such as chip selectsignals (via interface 3601) and information in row address fields togenerate one or more chip select signals from interface 3611 to aplurality of integrated circuit memory devices. For example, two one-hotchip select signals received at interface 3601 along with two bit valuesin a row address field may be used to output eight chip select signalsat interface 3611 to eight integrated circuit memory devices. Similarly,four received chip select signals may be used with one bit value in arow address field to output eight chip select signals from interface3611.

In an embodiment, information in a bank address field is used to outputchip select signals. Buffer device 3600 outputs chip select informationfrom interface 3611 to one or more integrated circuit memory devices inresponse to the bank address information received at interface 3601.Unused bank address fields/pins at interface 3601 may be used to providechip select information at interface 3611. For example, interface 3601may have 5 bank address pins while four integrated circuit memorydevices having 8 banks each are coupled to interface 3611. The lower 3pins, BA[2:0], would identify a particular bank in a particular memorydevice while the upper two bits BA[4:3] are used to decode/output chipselect signals. The four memory devices and buffer device 3600 then mayemulate one large memory die with 32 memory banks rather than 4 memorydies having 8 banks each.

In an embodiment, multiple chip select signals may be outputsimultaneously from interface 3611 to multiple respective memory devicesin response to information in a row address field, chip selectinformation and/or bank address information, singly or in combination.

Address translation circuit 3608 includes one or more multiplexers toreceive (via interface 3601) information in a row address field, chipselect information and/or bank address information and outputs signalsto interface 3611 that in turn outputs chips select signals in anembodiment.

One or more column address bit values may be re-tasked/remapped bybuffer 3600 to perform time slicing, as described above, in anembodiment. For example, the functions (or portions thereof) of datawidth translator 2950 may be performed by address translator 3608,command decode 3607, data path 3606 and data path router 3610, singly orin combination. Also, bit values in a column address field may also beused to initiate memory device functions/operations. When information ina column address field are re-tasked and this re-tasking uses lowerorder bit values, the remaining address bit values may be shifted tofill the lowest order column address bit values output at interface3611. For example, when bit values in column address A[4:3] in a columnaddress field are remapped to time slice address bits, column addressvalues in column address A[15:5] are shifted to column address A[13:3]to fill the lowest order column address bits.

In an embodiment, column address bit values may not be shifted whencolumn address bit values are used to initiate a memory deviceoperation. For example, a bit value in column address A[10] may be usedto trigger an auto-precharge operation in a DDR3 memory device. Whentime slicing is used as described above, a bit value in column addressbit A[10] would be mapped to column address bit A[10] (or not changed)while bit values in column addresses A[15:11] and A[9:5] are shifted tofill the gap caused by re-tasking bit values in column address A[4:3].Another similar example of not shifting a particular column addressvalue includes a bit value at column address A[12] used to trigger burstchop on column address cycles in a DDR3 memory device. In a burst chopmode of a DDR3 memory device, a portion of the read data (for examplethe last 4 bits of 8 bit output data) is masked or not output from anintegrated circuit memory device.

Buffer device 3600 may remap column bit values used to initiate a memorydevice operation (i.e., auto-precharge, burst chop, read sequenceordering) to particular column address bit fields. For example, bitvalues in column address bits A[2:0] are used to define bit orderingfrom a DDR memory device. Data on each signal line coupled to anintegrated circuit memory device will be returned in a different orderdepending on the column bit values at column address bits A[2:0]. Whenbuffer device 3600 performs time slicing, these column bit values arereassigned to a different value to match a “time” address used to storedata and to efficiently move data from an integrated circuit memorydevice to buffer device 3600. In an embodiment, data path 3606rearranges the data (from data path router 3610) in response to controlsignals from address translation circuit 3608 which receives column bitaddress values at column address A[2:0].

When less data is needed by buffer device 3600 than expected by anintegrated circuit memory device, such as in time slicing, the bufferdevice 3600 may use burst chop to save I/O power from the integratedcircuit memory devices. This would be irrelevant of the value of acolumn address bit A[12] (BCN). The received BCN bit values may bestored in the data path 3605 or command decode circuit 3607 that outputssignals to chop the data as originally requested by way of interface3601.

In an embodiment, received chip select information and bit values in areceived row address fields may be used by buffer device 3600 toassign/remap column bit values in column addresses output at interface3611.

Address translation circuit 3608 includes one or more multiplexers toreceive (via interface 3601) information in a column address fields andreassign/re-task column address bit values during time slicing and/orotherwise as describe above.

Buffer device 3600 may receive row address values or chip selectinformation that then may be used to configure a memory system thataccesses different sized/capacity (address space) memory modules duringdifferent modes of operations as described above in regard to FIGS.25-29. For example, row address values or chip select information may beused to select whether particular signal path widths are used inaccessing different sized memory modules during different modes ofoperation as illustrated in FIGS. 25A-B. In another example, row addressvalues or chip select information may be used to configure bypasscircuit 2900 shown in FIG. 29, such as enabling or disabling bypasspaths (i.e. via bypass elements 2905-2910) as well as selecting delaymultiplexers (i.e. outputting appropriate DELAY[0:3] control signals)shown in FIG. 29.

In embodiments, buffer 3600 may include JTAG 3603 and/or 12C 3604interfaces/circuits for accessing bit values in register set 3605. JTAG3603 may include a port having test pins used during testing of buffer3600. An 12C 3604 may be used for outputting or receiving bit values (byway of an 12C bus) for register set 3605 that outputs control signals tobuffer device circuit components in response to stored bit values thatmay represent particular buffer configurations. In an embodiment, bitvalues in register set 3605 may be accessed (written/read) directlythrough interface 3601.

In an embodiment, register set 3605 corresponds to configurationregister set 1881 shown in FIG. 18. In an embodiment, registers set 3605stores one or more bit values that indicate memory system topology sothat interface 3611 may be configured accordingly. For example, registerset 3605 may include bit values that indicate a number of integratedcircuit memory devices selected for a received memorytransaction/operation. Buffer device 3600 then may configure interface3611 (in response to register value) in order to match the bandwidthassociated with interface 3601.

In an embodiment, register set 3605 may store one or more bit valuesindicating where to obtain information in received control information(i.e. a request packet) that may be used in determining/remapping andoutputting chip select information or signals to one or more integratedcircuit memory devices. As described below, information in row addressfields, column address fields, bank address fields as well as receivedchip select signals may be used to decode and output predetermined chipselect signals from integrated circuit buffer device 3600 to theplurality of integrated circuit memory devices.

In an embodiment, register set 3605 may store one or more bit values toindicate a number of signal paths (i.e. width), type of signal pathtopology, a number of signal lines per signal path and/or a number (orexistence) of data signal strobe signal lines between integrated circuitbuffer device 3611 (in particular interface 3611) and a plurality ofintegrated circuit memory devices.

In an embodiment, register set 3605 may store one or more bit values toindicate how received column, row and/or bank addresses are reorderedand output from buffer device 3600.

PLL 3602 is used to synchronize the timing of receiving and/ortransmitting read and write data both internally and externally tobuffer 3600. In alternate embodiments, PLL 3602 may be another clockalignment circuit that corresponds to clock circuit 1870 shown in FIG.18. In an embodiment, PLL 3602 outputs WCLK and RCLK clock signals inresponse to a clock source that may be provided to buffer 3600.

FIGS. 37A-B illustrate timing diagrams for an integrated circuit bufferdevice. In particular, FIG. 37A illustrates a timing chart 3700 thatidentifies when a buffer device, such as buffer device 3600, receivesand outputs control/address information as well as receives and outputsread data when using a shared or command data signal path.

Control information, such as commands to activate a memory rank areillustrated by a shaded block A_(n) that represents the amount of timecontrol signals are provided on a control/address signal path (external(Ext.) RQ or internal (Int.) RQ signal paths) during cycles of a Clocksignal. For example, shaded block A_(a) on a row labeled Ext. RQrepresents a buffer device receiving a command to activate a memory rank“a” on an Ext. RQ signal path during a first clock cycle of the Clocksignal. Similarly a command to read a particular memory bank isillustrated by shaded blocks R_(n) on signal paths Ext. RQ and Int. RQ.For example, timing chart 3700 illustrates how a read command R_(a) isreceived by a buffer device via signal path Ext. RQ and a command R_(a)is output a clock cycle later onto signal path Int. RQ. In alternateembodiments, more or less memory commands or control signals may bereceived and generated.

Similarly, read data transferred on signal paths Ext. DQ and Int. DQ toa memory controller or from a memory rank are illustrated by a shadedblock labeled Read Data_(n). Write data may be similarly transferred.

Signal path Ext. RQ refers to a signal path that providescontrol/address information from a memory controller to the bufferdevice. Signal path Int. RQ refers to a signal path that providescontrol/address information from the buffer device to a plurality ofintegrated circuit memory devices or memory rank. Signal path Ext. DQrefers to a signal path that provides Read Data_(n) from the bufferdevice to the memory controller. Signal path Int. DQ refers to a signalpath that provides Read Data_(n) from a plurality of integrated circuitmemory devices or memory rank to the buffer device. In an embodiment,Ext. RQ corresponds to signal path 121 and Int. RQ corresponds to signalpath 1005; while Ext. DQ corresponds to signal path 120 a and Int. DQcorresponds to signal path 1006.

Timing chart 3700 illustrates that when memory ranks are coupled to thesame (or shared/common) signal path that transfers Read Data_(n), amemory system may have to be more complicated and less efficient. Inparticular, a shared signal path among memory ranks for transferringRead Data_(n) may require a memory controller to track accesses tomemory ranks and insert bubbles when changing access to different memoryranks. A “bubble” or “time bubble” refers to an amount of idle time amemory controller must insert in transferring data when switchingbetween memory transaction to the same memory rank. For example, amemory controller may have to insert a bubble or idle time whenswitching from accessing different memory ranks so as to allow theshared or common bus to settle (or allow time for tri-state drivers in atransceiver to switch to an alternate state as well as allow time foranother preamble signal) or for noise to dissipate before initiatinganother memory rank access or (in the case of strobed memory devices) toallow for a strobe preamble. This insertion of bubbles reduces signalpath utilization and may lower bandwidth on both internal and externalsignal paths.

FIG. 37B illustrates a timing chart 3701 that eliminates the need for amemory controller to track memory rank accesses and insert bubblesthereby reducing memory controller complexity and increasing bandwidth.Timing chart 3701 is similar to timing chart 3700 except rather thanhaving a shared signal path for transferring data between a bufferdevice and memory ranks, segmented signal paths or dedicated signalpaths Int. DQ(0)-(7) are provided between the buffer device and eachmemory rank (8 memory ranks). Bubbles are no longer present on the Ext.DQ signal path as Read Data_(a-f) are provided on separate signal pathsInt. DQ(0)-(7) from respective memory ranks.

FIG. 38 illustrates a system 3800 including a buffer device 3600 and aplurality of integrated circuit memory devices 101 a-101 n organized indifferent memory ranks (1-4). System 3800 may be included in a memorysystem including other buffer devices and/or memory controllers asdescribed herein.

A “memory rank” or “rank” refers to a number of integrated circuitmemory devices grouped to output a predetermined amount of data bits orblocks of data, such as 72 data bits (64 data bits plus 8 ECC bitsprovided by an ECC device), onto a signal path during a predeterminedperiod of time. For example a dual rank system (using rank 1 and rank 2shown in FIG. 38) may provide two 64 data bit blocks from two sets ofintegrated circuit memory devices, rank 1 and rank 2. In an embodiment,the integrated circuit memory devices may be ×4 memory devices (memorydevices that produce 4 bits of data) or ×8 memory devices (memorydevices that produce 8 bits of data). In this example, 8×8 memorydevices could produce a 64 data bit block or 16×4 memory devices couldproduce a 64 data bit block. In embodiments, different numbers of ranksmay be used.

Buffer device 3600 receives control/address information as well as datafrom a memory controller via signal paths 120 a and 121. In anembodiment, interface 3601 as illustrated in FIG. 36, is used to receivecontrol/address information and write data as well as output read datafrom integrated circuit memory devices in system 3800. Buffer device3600 outputs translated (and/or decoded) control/address information aswell as selected write data to integrated circuit memory devices 101 a-nin memory ranks 1-4 using interface 3611 of buffer 3600.

Interface 3611 is coupled to signal paths 3801-3804 and signal path3810. Signal paths 3801-3804 are segmented signal paths to transfer readand write data between buffer device 3600 and integrated circuit memorydevices in ranks 1-4. Signal path 3801 is coupled to memory devices 101a-n in rank 1. Signal path 3802 is coupled to memory devices 101 a-n inrank 2. Signal path 3803 is coupled to memory devices 101 a-n in rank 3.Signal path 3804 is coupled to memory devices 101 a-n in rank 4. In anembodiment, read and write data is transferred using a segmentedtopology as illustrated in FIG. 34.

In contrast, signal path 3810 provides control/address information tomemory ranks 1-4 on a shared/common signal path 3810, such as a fly-bytopology shown in FIG. 33A. Each memory device in each memory rank iscoupled to shared signal path 3810. In embodiments, clock signals orclock information may be provided on either signal paths 3801-3804 orsignal path 3810 or on another separate signal path.

FIG. 39 illustrates a system 3900 for accessing individual memorydevices that function as respective memory ranks. System 3900illustrates an embodiment similar to system 3800 except that memorydevices 3901 a-h are included in respective memory ranks. In anembodiment, memory devices 3900 a-h are eight×4 DDR3 memory devices.Accordingly, system 3900 is an eight rank system having respectivesegmented data signal paths. Segmented signal paths 3904 a-h transferdata bits DQ [0:3] between data segment (segmentation) and merge circuit3902 and respective memory devices 3901 a-h. A data mask signal DM isprovided to respective memory devices 3901 a-h from data segment andmerge circuit 3902. Similarly, clock signals or differential strobesignals DQS and DQSN are provided from data segment and merge circuit3902 for synchronization of data signals. Control/address signals areprovided on signal path 3903 that is a shared signal path similar tosignal path 3810 shown in FIG. 38.

In an embodiment, data segment and merge circuit 3902 operates similarto one or more circuit components in buffer device 3600 shown in FIG.36. Data segment and merge circuit 3902 merges read data from aplurality of memory devices 3901 a-h onto a single signal path as a readdata stream. Likewise, data segment and merge circuit 3902 segments asingle write data from a single signal path into multiple write dataoutput to multiple signal paths coupled to multiple memory devices 3901a-h. For example, data segment and merge circuit 3902 may include thefunctionality of data path circuit 3606, data path router 3610, commanddecode 3607 and address translation circuit 3608, singly or incombination. In an embodiment, mux control and RQ state information isprovided by a control circuit, such as command decode 3607 and addresstranslation circuit 3608 shown in FIG. 36. Mux control and RQ stateinformation determines the source or destination of read/write data.

FIG. 40 illustrates a method 4000 of operation in an integrated circuitbuffer device. In an embodiment, buffer device 3600 performs method4000. Method 4000 begins at logic block 4001 where an integrated circuitbuffer device is reset and/or power is provided. In logic block 4002, anintegrated circuit buffer device receives first control information thatindicates a read operation for a first memory organization. In anembodiment, a master provides the first control information to access afirst memory configuration that includes a first predetermined number ofmemory devices, banks as well as predetermined page length/size andbandwidth. However, the buffer device interfaces with a second differentmemory organization that may include a second predetermined number ofmemory devices, banks as well as predetermined page length/size andbandwidth.

A virtual page size/length may be the size of data or memory block thatmay be used by a processor or memory controller. For example, if aprocess requests an operating system to allocate 64 bytes, but the pagesize is 4 KB, then the operating system must allocate an entire virtualpage or 4 KB to the process. In embodiments, a physical page size/lengthmay equal the amount of data provided by a memory rank or the amount ofdata bits available from a plurality of sense amplifiers in one or morebanks of one of more integrated circuit memory devices in the memoryrank. A virtual page size may equal a physical page size in anembodiment. A memory controller may be able to adjust the virtual pagesize but not the physical page size.

Logic blocks 4003 and 4004 illustrate outputting second and thirdcontrol information to a first signal path coupled to first and secondintegrated circuit memory devices in the second memory organization.

Logic blocks 4005 and 4006 illustrate receiving first and second datafrom second and third signal paths coupled to the first and secondintegrated circuit memory devices in the second memory organization.

Logic block 4007 illustrates merging and output read data that includesthe first and second read data from the integrated circuit buffer devicein response to the first control information.

In an embodiment, one or more logic blocks 4002-4007 may be repeated.

Logic block 4008 illustrates ending method 4000 when power is removed.In alternate embodiments, method 4000 may end without power removed.

A method of operation of a buffer device that transfers write dataperforms similar steps illustrated in method 4000. However rather thanreceiving and outputting read data as illustrated by blocks 4005-4007,write data may be segmented and transferred to second and third signalpaths in response to first control information.

Signals described herein may be transmitted or received between andwithin devices/circuits using signal paths and generated using anynumber of signaling techniques including without limitation, modulatingthe voltage or current level of an electrical signal. The signals mayrepresent any type of control and timing information (e.g. commands,address values, clock signals, and configuration/parameter information)as well as data. In an embodiment, a signal described herein may be anoptical signal.

A variety of signals may be transferred on signal paths as describedherein. For example, types of signals include differential (over a pairof signal lines), non-return to zero (“NRZ”), multi-level pulseamplitude modulation (“PAM”), phase shift keying, delay or timemodulation, quadrature amplitude modulation (“QAM”) and Trellis coding.

In an embodiment employing multi-level PAM signaling, a data rate may beincreased without increasing either the system clock frequency or thenumber of signal lines by employing multiple voltage levels to encodeunique sets of consecutive digital values or symbols. That is, eachunique combination of consecutive digital symbols may be assigned to aunique voltage level, or pattern of voltage levels. For example, a4-level PAM scheme may employ four distinct voltage ranges todistinguish between a pair of consecutive digital values or symbols suchas 00, 01, 10 and 11. Here, each voltage range would correspond to oneof the unique pairs of consecutive symbols.

In an embodiment, a clock signal is used to synchronize events in amemory module and/or device such as synchronizing receiving andtransmitting data and/or control information. In an embodiment, globallysynchronous clocking is used (i.e., where a single clock frequencysource is distributed to various devices in a memory module/system). Inan embodiment, source synchronous clocking is used (i.e., where data istransported alongside a clock signal from a source to a destination suchthat a clock signal and data become skew tolerant). In an embodiment,encoding data and a clock signal is used. In alternate embodiments,combinations of clocking or synchronization described herein are used.

In embodiments, signal paths described herein include one or moreconducting elements, such as a plurality of wires, metal traces(internal or external), signal lines or doped regions (positively ornegatively enhanced), as well as one or more optical fibers or opticalpathways, singly or in combination. In embodiments, multiple signalpaths may replace a single signal path illustrated in the Figures and asingle signal path may replace multiple signal paths illustrated in theFigures. In embodiments, a signal path may include a bus and/orpoint-to-point connection. In an embodiment, signal paths include signalpaths for transferring control and data signals. In an alternateembodiment, signal paths include only signals paths for transferringdata signals or only signal paths for transferring control signals. Instill other embodiments, signal paths transfer unidirectional signals(signals that travel in one direction) or bidirectional signals (signalsthat travel in two directions) or combinations of both unidirectionaland bidirectional signals.

It should be noted that the various circuits disclosed herein may bedescribed using computer aided design tools and expressed (orrepresented) as data and/or instructions embodied in variouscomputer-readable media, in terms of their behavior, register transfer,logic component, transistor, layout geometries, and/or othercharacteristics. Formats of files and other objects in which suchcircuit expressions may be implemented include, but are not limited to:formats supporting behavioral languages such as C, Verilog, and HLDL;formats supporting register level description languages like RTL;formats supporting geometry description languages such as GDSII, GDSIII,GDSIV, CIF, MEBES; and any other suitable formats and languages.Computer-readable media in which such formatted data and/or instructionsmay be embodied include, but are not limited to, non-volatile storagemedia in various forms (e.g., optical, magnetic or semiconductor storagemedia) and carrier waves that may be used to transfer such formatteddata and/or instructions through wireless, optical, or wired signalingmedia or any combination thereof. Examples of transfers of suchformatted data and/or instructions by carrier waves include, but are notlimited to, transfers (uploads, downloads, e-mail, etc.) over theInternet and/or other computer networks via one or more data transferprotocols (e.g., HTTP, FTP, SMTP, etc.). When received within a computersystem via one or more computer-readable media, such data and/orinstruction-based expressions of the above described circuits may beprocessed by a processing entity (e.g., one or more processors) withinthe computer system in conjunction with execution of one or more othercomputer programs including, without limitation, netlist generationprograms, place and route programs and the like, to generate arepresentation or image of a physical manifestation of such circuits.Such representation or image may thereafter be used in devicefabrication, for example, by enabling generation of one or more masksthat are used to form various components of the circuits in a devicefabrication process.

The foregoing description of several embodiments has been provided forthe purposes of illustration and description. It is not intended to beexhaustive or to limit the embodiments to the precise forms disclosed.Modifications and variations will be apparent to practitioners skilledin the art. The embodiments were chosen and described in order toexplain inventive principles and practical applications, therebyenabling others skilled in the art to understand various embodiments andwith the various modifications as are suited to the particular usecontemplated. It is intended that the scope of the invention be definedby the following claims and their equivalents.

1. (canceled)
 2. An integrated circuit device comprising: an interfaceto communicate with a first plurality of stacks of memory dies, eachmemory die of each stack having a respective array of memory cells; andthe interface to transmit to the first plurality of stacks a first chipselect signal to select each stack of the first plurality of stacks forthe memory access, and a memory access command that specifies the memoryaccess, the memory access command including information to select aparticular one of the memory dies within each stack of the firstplurality of stacks for the memory access.
 3. The integrated circuitdevice of claim 2 wherein the interface is to transmit a packet to thefirst plurality of stacks, the packet comprising the memory accesscommand and an address, the address to identify a memory location in theparticular one of the memory dies within each of the first plurality ofstacks for the memory access.
 4. The integrated circuit device of claim3 wherein the information is part of the address.
 5. The integratedcircuit device of claim 4 wherein the information is part of a bankaddress field of the address.
 6. The integrated circuit device of claim2 wherein the interface is to transmit to a second plurality of stacksof memory dies a second chip select signal, to select each stack of thesecond plurality of stacks for the memory access.
 7. The integratedcircuit device of claim 2 further comprising an interface to transferdata between the integrated circuit device and the particular one of thememory dies within each of the first plurality of stacks for the memoryaccess.
 8. The integrated circuit device of claim 2 wherein theintegrated circuit device is a memory controller device.
 9. Theintegrated circuit device of claim 6 wherein the first plurality ofstacks is disposed on a memory module comprising a serial presencedetect (SPD) device and wherein the integrated circuit is to access theSPD device to retrieve configuration information relating to the firstplurality of stacks.
 10. The integrated circuit device of claim 2wherein array respective array of memory cells is an array of dynamicrandom access memory (DRAM) cells and wherein the integrated circuit isa DRAM memory controller integrated circuit.
 11. A integrated circuitdevice comprising: an interface to communicate with at least one stackof memory dies, each memory die of the at least one stack having arespective array of memory cells; the interface to transmit a chipselect signal to select a given stack of the at least one stack for amemory access, and a memory access command that specifies the memoryaccess, the memory access command including information to select aparticular die within the given stack for the memory access, and amemory location within the array respective to the particular die forthe memory access.
 12. The integrated circuit device of claim 11 whereinthe interface is to transmit a packet, the packet comprising the memoryaccess command and an address, the address to identify a memory locationin the respective array of the particular die for the memory access. 13.The integrated circuit device of claim 12 wherein the integrated circuitdevice is to transmit an address in association with the memory accesscommand and wherein the information is part of the address.
 14. Theintegrated circuit device of claim 11 wherein: the given stack comprisesa buffer die; the interface is to be coupled to the buffer die; and thebuffer die is to receive the information from the interface and is toretransmit the memory access command to the particular die.
 15. Theintegrated circuit device of claim 14 wherein the information is part ofa bank address field of the address.
 16. The integrated circuit deviceof claim 11 wherein: the given stack has a buffer die; the interface isto be coupled to the buffer die; the buffer die is to generate a secondchip select signal, internal to the given stack, to select theparticular die; and the buffer die is to retransmit the memory accesscommand for the given stack to the particular die of the given stack.17. The integrated circuit device of claim 16 wherein: the integratedcircuit device further comprises an interface to transfer data betweenthe integrated circuit device and the buffer die; and the buffer die isto transfer the data between a memory controller integrated circuit andthe particular die of the given stack.
 18. The integrated circuit deviceof claim 16 wherein: the interface is a command interface; theintegrated circuit device further comprises a data interface to transferdata between the integrated circuit device and the given stack; thegiven stack is to store a slice of data; the data interface is not totransfer the slice of data between the integrated circuit device and thebuffer die in association with the memory access command; and the datainterface is to transfer the slice of data between the integratedcircuit device and the particular die of the given stack.
 19. A methodof operation of an integrated circuit device having an interface tocommunicate with a first plurality of stacks of memory dies, each memorydie of each stack having a respective array of memory cells, the methodcomprising: transmitting to the first plurality of stacks, a first chipselect signal to select each stack of the first plurality of stacks forthe memory access; and transmitting to the first plurality of stacks, inassociation with the first chip select signal, a memory access commandthat specifies the memory access, the memory access command includinginformation to select a particular one of the memory dies within eachstack of the first plurality of stacks for the memory access.
 20. Themethod of claim 19 further comprising transmitting to a second pluralityof stacks of memory dies, a second chip select signal to select eachstack of the second plurality of stacks for the memory access.
 21. Themethod of claim 20 wherein the integrated circuit device is a memorycontroller device.